Caching Strategies: Why Invalidation Is the Hard Part (2026)

Adding a cache is a one-afternoon job: drop Redis in front of the slow query, watch the latency graph fall, feel like a genius. The bill comes later, when someone updates a record and the app keeps serving the old value for six hours — and nobody can explain why.

That’s the real shape of caching strategies. Choosing where to cache and which pattern to use is the easy 20%. The hard 80% is invalidation: making sure the copy you sped everything up with doesn’t quietly become a lie. There’s a reason Phil Karlton’s line — quoted on Martin Fowler’s Two Hard Things — has survived for decades: “There are only two hard things in Computer Science: cache invalidation and naming things.”

This post is part of the System Design Foundations series, anchored by the System Design Building Blocks overview. By the end you’ll know the caching patterns, how eviction works, and — the part most guides skip — how to reason about staleness before it burns you.

Why You Cache — and What It Costs#

A cache is a small, fast copy of data kept close to whoever needs it, so you don’t redo expensive work — a slow query, a remote API call, a heavy computation. As AWS’s caching overview puts it, the win is lower latency and less load on the thing behind the cache. Done well, a cache turns a 200 ms database hit into a 2 ms memory read and takes pressure off your database at the same time.

But you haven’t removed the truth — you’ve duplicated it. And the instant there are two copies of anything, they can disagree. Every caching decision is really a bet: this data won’t change often enough for a slightly-stale copy to hurt. When that bet is right, caching is the cheapest performance win in the building. When it’s wrong, you ship stale data to users and spend an afternoon figuring out which layer lied.

Where Caching Lives: The Layers#

Caching isn’t one thing in one place — a request passes through several, each a chance to answer early.

Read that left to right. The browser caches so it needn’t ask at all. A CDN caches static assets near the user. An application cache like Redis holds hot query results and computed values. Behind them all sits the database — the slowest to reach, but the one source that’s always right. The further right a request travels, the slower and more authoritative the answer. The whole art of caching is answering as far left as you safely can — without the left-hand copies going stale.

The Core Caching Patterns#

“Caching strategy” usually means one of these read/write patterns. They differ in who moves data in and out, and when.

Cache-aside — the one you’ll write most — is just four steps in code:

# Cache-aside: the app owns the cache. Check it first, fall back to the DB.
def get_user(user_id):
    user = cache.get(user_id)          # 1. look in the cache
    if user is not None:
        return user                    # 2. hit — return it, no DB call
    user = db.query(user_id)           # 3. miss — read the source of truth
    cache.set(user_id, user, ttl=300)  # 4. populate the cache for next time
    return user

It’s the default for a reason: it’s simple and it degrades gracefully (if the cache is down, you just hit the DB). The write patterns are really a dial between freshness and write speed: write-through keeps the cache honest at the cost of latency; write-behind is fast but risks losing not-yet-flushed data. None of them make staleness go away — they just move where you pay for it.

Eviction: What to Throw Away#

A cache is small on purpose, so it has to forget things. The eviction policy decides what goes when it fills up. LRU (least recently used) is the sensible default — throw out what hasn’t been touched in the longest time. LFU (least frequently used) keeps the popular keys even if they’ve gone briefly quiet. FIFO just evicts the oldest. And TTL (time to live) evicts on a clock rather than on pressure — which is where eviction blurs into invalidation, because a TTL is often the only thing standing between your cache and stale data.

The Hard Part: Invalidation#

Here’s the 80%. A cache is a copy, and when the real data changes, the copy is wrong until you fix it. You have exactly two ways to fix it, and both bite.

The first is a TTL — let entries expire after N seconds. It’s simple and needs no coordination, but it’s a guess. Too long and you serve stale data for the whole window; too short and you’re barely caching at all. The second is explicit invalidation — delete or update the cached key the moment the underlying data changes. It’s precise when it works, but now every code path that writes the data has to remember to touch the cache, and the one path that forgets fails silently, serving stale reads until someone notices. And the path that forgets looks completely innocent — it’s usually one missing line:

# On write, update the DB *and* invalidate the cached copy.
def update_user(user_id, changes):
    db.update(user_id, changes)
    cache.delete(user_id)   # drop this line and every reader gets stale data

Then there’s the failure that only shows up under load: the cache stampede. A hot key expires, hundreds of requests miss at the same instant, and they all slam the database to recompute the same value at once — often taking the database down exactly when traffic is highest. The fixes are a lock so only one request recomputes, refreshing just before expiry, or staggering TTLs so keys don’t all die together.

One pattern worth knowing: in a system already running an event stream, invalidation gets much cleaner. Publish a “this record changed” event and let every service drop the affected key — event-driven invalidation instead of hoping each write path remembers. It’s a natural fit for the log-shaped systems in the Kafka vs RabbitMQ post.

When NOT to Cache#

Caching is a reflex, and reflexes over-fire. Skip the cache when:

• The data is write-heavy and read-light — you’ll spend more effort invalidating than you ever save on reads.
• Correctness is non-negotiable — account balances, inventory counts, anything where a stale read causes a real-world error. Read the source.
• It’s cheap to compute anyway — if the “expensive” operation is a fast indexed lookup, a cache just adds a second thing to keep in sync.
• Every request is unique — if nothing is ever read twice, your hit rate is zero and you’ve added complexity for nothing.

Caching in the Age of LLMs#

Caching matters more for LLM apps than almost anywhere else, because the thing you’re caching — a model call — is slow and expensive per request. But the AI versions of caching also drag the invalidation problem into sharper focus.

Exact-match and prompt caching are the easy win. If two users send the identical prompt, you can return a stored response instead of paying for inference again. Providers extend this with prompt caching that reuses the computed state of a long, unchanging system prompt across calls — cutting both latency and cost. This is just “cache the result,” applied to a result that happens to cost real money.

Semantic caching is where invalidation gets harder, not easier. Instead of matching exact strings, a semantic cache embeds the query and returns a stored answer when a previous query is close enough by cosine similarity — so “What is RAG?” and “Can you explain retrieval-augmented generation?” hit the same cached answer. The catch is the threshold: set it too loose and the cache confidently serves an answer to a subtly different question. Recent research on calibrating semantic caches shows that static similarity thresholds don’t give correctness guarantees — which is Phil Karlton’s warning in a new costume. And a cached answer to a time-sensitive question goes stale silently, so semantic caches still need short TTLs or event-driven invalidation on the underlying data.

Cache embeddings, not just answers. In a RAG pipeline, re-embedding the same document or query is pure waste — embeddings are deterministic for a given model, so they cache perfectly. Storing them (the job of your vector database) means you never pay to embed the same text twice; the embeddings primer covers why that’s safe to do.

Quick Recap#

• A cache is a faster copy of the truth that trades freshness for speed.
• Patterns — cache-aside, read/write-through, write-behind, write-around — are a dial between freshness and write cost.
• Eviction (LRU/LFU/TTL) decides what to forget; invalidation decides when the copy is wrong.
• Invalidation is the hard part: TTLs guess, explicit invalidation is easy to miss, and stampedes bite under load.
• In AI systems, exact-match and prompt caching are easy wins; semantic caching makes staleness riskier; embeddings cache for free.

Conclusion#

The whole of caching strategies comes down to one uncomfortable truth: the moment you make a second copy of your data, you own the problem of keeping it honest. Picking cache-aside over write-through is a five-minute decision. Deciding how stale you can afford to be, and building the invalidation to enforce it, is the actual work — and it’s the part that separates a cache that speeds you up from one that ships quiet bugs. Add the copy on purpose, know exactly how it gets corrected, and never cache what you can’t afford to be wrong about.

What’s the worst stale-cache bug you’ve shipped — a TTL set too long, an invalidation path someone forgot, or a stampede that took the database down? I want to hear the war story — the comments are open.