Cache

Updated Nov 16, 2026 · 6 min read
On this page

    Your product page loads in 400 ms. Profiling shows the database answers the same query, "give me product #42", thousands of times a minute, and every answer is identical. The database is busy doing work it already did a second ago. During a flash sale the read rate triples, CPU on the database hits 95%, and every page slows down, including checkout.

    The fix is to keep a copy of the answer somewhere much faster than the database and serve repeats from there. That copy is a cache.

    How it works

    The service sits between the request and the database and asks the cache first.

    Cache-aside read path: the Product Service checks Redis for product:42; on a hit it returns in about 1 ms, on a miss it reads the database and writes the value back with a TTL.
    Cache-aside read path: the Product Service checks Redis for product:42; on a hit it returns in about 1 ms, on a miss it reads the database and writes the value back with a TTL.
    1. A request for product #42 arrives through the API Gateway at the Product Service.
    2. The service runs GET product:42 against Redis. If the value is there (a hit), it returns it. Done in about a millisecond.
    3. If it is not there (a miss), the service reads the row from the database, writes it into Redis with SET product:42 … EX 600, and returns it.
    4. The next thousand readers get a hit.

    This is cache-aside (also called lazy loading): the application decides what goes into the cache, and only data that someone actually asked for is ever cached. It is the default you should reach for in an interview.

    On a write, the service updates the database first and then deletes the cache key. The next read misses and loads the fresh value. Deleting is safer than updating the cache in place: if two writers race, an update can leave the older value in the cache, while a delete just forces one extra miss.

    Other write strategies

    StrategyHow a write worksGood forWatch out for
    Cache-aside + deleteWrite DB, then delete the keyMost read-heavy servicesA short window where the old value can still be read
    Write-throughWrite cache and DB together, in the requestData read right after it is writtenEvery write pays both latencies; cache fills with data nobody reads
    Write-behindWrite cache now, flush to DB laterVery high write rates (counters, likes)Data loss if the cache node dies before the flush
    Refresh-aheadReload hot keys before they expireA few very hot keysExtra load for keys that turn cold

    Expiry and eviction

    Two separate mechanisms remove entries:

    • TTL (time to live): each key expires after a set time, for example 10 minutes. TTL is your upper bound on staleness. If a price changes and you forget to invalidate, readers see the old price for at most one TTL.
    • Eviction: memory is finite. When the cache is full it removes keys by a policy, usually LRU (least recently used) or LFU (least frequently used). Eviction is why a cache works with a small fraction of the data: you only need the hot set.

    A useful rule of thumb: a cache sized for the top 10–20% of keys often serves 80% or more of reads, because popularity is skewed.

    Trade-offs and costs

    • Staleness. Every cached value can be out of date. You trade freshness for speed. Ask: how long can a reader see an old value before it hurts? Product descriptions: minutes. Account balance: never.
    • Another system to run. Memory costs more per GB than disk, the cluster needs monitoring, and a cache outage moves all traffic back to the database at once.
    • Invalidation bugs. The hard part is not reading the cache; it is knowing every place that changes the data and making sure each one deletes the right keys.
    • Cold start. After a deploy or a cache restart, the hit rate is zero and the database takes full load until the cache warms.
    • Hot keys. One key read by everyone (a celebrity's post, a flash-sale product) can overload a single cache node, because one key lives on one shard.

    When a hot key expires: the stampede

    Cache stampede: without protection, every request that misses on the expired key queries the database at once; with a rebuild lock, one request reloads the value while the others wait briefly and then
    Cache stampede: without protection, every request that misses on the expired key queries the database at once; with a rebuild lock, one request reloads the value while the others wait briefly and then read it from the cache.

    If product #42 gets 1,000 reads per second and its key expires, every request in the next few milliseconds misses and runs the same query. The database gets a burst of identical work exactly when it can least afford it. Three standard fixes:

    • Rebuild lock (single flight): the first request to miss takes a short lock (SET lock:product:42 NX EX 5), reloads the value and fills the cache. Others wait a few milliseconds and read again.
    • Jittered TTLs: add a random 0–10% to each TTL so keys loaded at the same time do not expire at the same time.
    • Early refresh: when a hot key is close to expiry, refresh it in the background while still serving the old value.

    When to reach for it

    Reach for a cache when you can say yes to most of these:

    • Reads clearly outnumber writes (10:1 or more).
    • The same items are read again and again (skewed popularity).
    • A slightly stale answer is acceptable for some bounded time.
    • Reads are expensive: a slow query, a join, a call to another service.
    • The database is the bottleneck, or latency targets are tight (p99 under 100 ms).

    Think twice when:

    • Every request reads different data (no reuse means no hits).
    • The data must always be exact at the moment of reading (balances, inventory at checkout). Read those from the database.
    • Writes dominate. You would invalidate more often than you serve hits.
    • The system is small. A database with the right index may be fast enough; a cache adds moving parts.

    Example

    Take a URL shortener. Redirects (GET /abc123) outnumber new links by about 100 to 1, and a link never changes after it is created.

    • What is cached: short_code → long_url.
    • Key: url:abc123. Value: the long URL (a few hundred bytes).
    • Pattern: cache-aside. A miss reads the database and fills the cache.
    • TTL: 24 hours, or the link's own expiry time if it is shorter. Since mappings never change, there is almost nothing to invalidate; deleting a link deletes the key.
    • Sizing (assumption): 100 million links, the hot 20% at about 500 bytes each is about 10 GB. That fits on a small Redis cluster with a replica per shard.
    • Hot link: a link shared by a celebrity can get 50,000 reads per second. Put the redirect behind the CDN with a short cache time, or keep a small in-process cache (a few seconds) in each server so most reads never leave the machine.

    The result: the database sees mostly writes and the occasional miss, and redirect latency drops from tens of milliseconds to a few.

    Common mistakes

    • "Add Redis" with no detail. Say what is cached, the key, the value, the TTL, and how it is invalidated.
    • Treating the cache as the source of truth. Caches lose data on restart and evict under memory pressure. The database must always be able to answer.
    • Caching data that must be exact. Reading stock levels at checkout from a cache sells items you do not have.
    • Updating the cache on write instead of deleting. Racing writers can leave the old value behind for a whole TTL.
    • Ignoring the stampede and hot keys. Interviewers often ask "what happens when this key expires during the sale?"
    • No plan for a cache outage. If the cache dies, can the database take full load? If not, you need replicas, rate limiting, or a degraded mode.

    Say this in the interview

    "Product reads are about 50 to 1 over writes and most traffic hits a few thousand popular items, so I'll put Redis in front of the product table with cache-aside. Key is product:{id}, value is the serialized product, TTL ten minutes with some jitter. On update we write the database and delete the key. For the flash-sale item I'll add a rebuild lock so an expiry doesn't send a thousand identical queries to the database. Stock at checkout is read from the database, not the cache."

    Used in