Cache
On this page
Your product page loads in 400 ms. Profiling shows the database answers the same query, "give me product #42", thousands of times a minute, and every answer is identical. The database is busy doing work it already did a second ago. During a flash sale the read rate triples, CPU on the database hits 95%, and every page slows down, including checkout.
The fix is to keep a copy of the answer somewhere much faster than the database and serve repeats from there. That copy is a cache.
How it works
The service sits between the request and the database and asks the cache first.

- A request for product #42 arrives through the API Gateway at the Product Service.
- The service runs
GET product:42against Redis. If the value is there (a hit), it returns it. Done in about a millisecond. - If it is not there (a miss), the service reads the row from the database, writes it into Redis with
SET product:42 … EX 600, and returns it. - The next thousand readers get a hit.
This is cache-aside (also called lazy loading): the application decides what goes into the cache, and only data that someone actually asked for is ever cached. It is the default you should reach for in an interview.
On a write, the service updates the database first and then deletes the cache key. The next read misses and loads the fresh value. Deleting is safer than updating the cache in place: if two writers race, an update can leave the older value in the cache, while a delete just forces one extra miss.
Other write strategies
| Strategy | How a write works | Good for | Watch out for |
|---|---|---|---|
| Cache-aside + delete | Write DB, then delete the key | Most read-heavy services | A short window where the old value can still be read |
| Write-through | Write cache and DB together, in the request | Data read right after it is written | Every write pays both latencies; cache fills with data nobody reads |
| Write-behind | Write cache now, flush to DB later | Very high write rates (counters, likes) | Data loss if the cache node dies before the flush |
| Refresh-ahead | Reload hot keys before they expire | A few very hot keys | Extra load for keys that turn cold |
Expiry and eviction
Two separate mechanisms remove entries:
- TTL (time to live): each key expires after a set time, for example 10 minutes. TTL is your upper bound on staleness. If a price changes and you forget to invalidate, readers see the old price for at most one TTL.
- Eviction: memory is finite. When the cache is full it removes keys by a policy, usually LRU (least recently used) or LFU (least frequently used). Eviction is why a cache works with a small fraction of the data: you only need the hot set.
A useful rule of thumb: a cache sized for the top 10–20% of keys often serves 80% or more of reads, because popularity is skewed.
Trade-offs and costs
- Staleness. Every cached value can be out of date. You trade freshness for speed. Ask: how long can a reader see an old value before it hurts? Product descriptions: minutes. Account balance: never.
- Another system to run. Memory costs more per GB than disk, the cluster needs monitoring, and a cache outage moves all traffic back to the database at once.
- Invalidation bugs. The hard part is not reading the cache; it is knowing every place that changes the data and making sure each one deletes the right keys.
- Cold start. After a deploy or a cache restart, the hit rate is zero and the database takes full load until the cache warms.
- Hot keys. One key read by everyone (a celebrity's post, a flash-sale product) can overload a single cache node, because one key lives on one shard.
When a hot key expires: the stampede

If product #42 gets 1,000 reads per second and its key expires, every request in the next few milliseconds misses and runs the same query. The database gets a burst of identical work exactly when it can least afford it. Three standard fixes:
- Rebuild lock (single flight): the first request to miss takes a short lock (
SET lock:product:42 NX EX 5), reloads the value and fills the cache. Others wait a few milliseconds and read again. - Jittered TTLs: add a random 0–10% to each TTL so keys loaded at the same time do not expire at the same time.
- Early refresh: when a hot key is close to expiry, refresh it in the background while still serving the old value.
When to reach for it
Reach for a cache when you can say yes to most of these:
- Reads clearly outnumber writes (10:1 or more).
- The same items are read again and again (skewed popularity).
- A slightly stale answer is acceptable for some bounded time.
- Reads are expensive: a slow query, a join, a call to another service.
- The database is the bottleneck, or latency targets are tight (p99 under 100 ms).
Think twice when:
- Every request reads different data (no reuse means no hits).
- The data must always be exact at the moment of reading (balances, inventory at checkout). Read those from the database.
- Writes dominate. You would invalidate more often than you serve hits.
- The system is small. A database with the right index may be fast enough; a cache adds moving parts.
Example
Take a URL shortener. Redirects (GET /abc123) outnumber new links by about 100 to 1, and a link never changes after it is created.
- What is cached:
short_code → long_url. - Key:
url:abc123. Value: the long URL (a few hundred bytes). - Pattern: cache-aside. A miss reads the database and fills the cache.
- TTL: 24 hours, or the link's own expiry time if it is shorter. Since mappings never change, there is almost nothing to invalidate; deleting a link deletes the key.
- Sizing (assumption): 100 million links, the hot 20% at about 500 bytes each is about 10 GB. That fits on a small Redis cluster with a replica per shard.
- Hot link: a link shared by a celebrity can get 50,000 reads per second. Put the redirect behind the CDN with a short cache time, or keep a small in-process cache (a few seconds) in each server so most reads never leave the machine.
The result: the database sees mostly writes and the occasional miss, and redirect latency drops from tens of milliseconds to a few.
Common mistakes
- "Add Redis" with no detail. Say what is cached, the key, the value, the TTL, and how it is invalidated.
- Treating the cache as the source of truth. Caches lose data on restart and evict under memory pressure. The database must always be able to answer.
- Caching data that must be exact. Reading stock levels at checkout from a cache sells items you do not have.
- Updating the cache on write instead of deleting. Racing writers can leave the old value behind for a whole TTL.
- Ignoring the stampede and hot keys. Interviewers often ask "what happens when this key expires during the sale?"
- No plan for a cache outage. If the cache dies, can the database take full load? If not, you need replicas, rate limiting, or a degraded mode.
Say this in the interview
"Product reads are about 50 to 1 over writes and most traffic hits a few thousand popular items, so I'll put Redis in front of the product table with cache-aside. Key is
product:{id}, value is the serialized product, TTL ten minutes with some jitter. On update we write the database and delete the key. For the flash-sale item I'll add a rebuild lock so an expiry doesn't send a thousand identical queries to the database. Stock at checkout is read from the database, not the cache."
Used in
- Read-heavy Systems: caching is the first tool for read-heavy designs.
- CDN: a cache at the edge, close to the user.
- TTL, Expiration & Cleanup: how expiry is implemented.
- Design a URL Shortener, Design a Distributed Cache like Redis, Design a Flash Sale System.