The application checks the cache; on a miss it reads the database, writes the value into the cache with a TTL and returns it. On updates it writes the database and invalidates the cache key.

Compare write-through and write-back caching.

Write-through writes the cache and database synchronously, keeping the cache fresh at the cost of slower writes. Write-back writes the cache first and persists later, giving fast writes but risking data loss if the cache fails.

What is a thundering herd or cache stampede?

When a popular key expires, many concurrent requests miss and hit the database together. Prevent it with request coalescing (single flight), early refresh, TTL jitter or serving stale data while one request refreshes.

What is LRU eviction?

Least Recently Used: when the cache is full, it removes the entry that has not been accessed for the longest time. LFU (least frequently used) is an alternative for skewed access.

Why is cache invalidation hard?

Copies of data exist in many places and updates race with reads, so a cache can be refilled with an old value after a delete. TTLs, versioned keys and careful ordering limit how stale data can get.

What happens if the cache goes down?

All reads fall through to the database, which may not handle the load. Size the database for a cold cache, warm the cache gradually, use replicas, or degrade non-critical features.

How do you handle a hot key?

Add an in-process local cache in front of Redis, replicate the key under several names and pick one at random, or use read replicas of the cache node.

What should never be cached, or cached carefully?

Rapidly changing or strongly consistent data like account balances and inventory counts at checkout. Personalised data must be keyed per user so one user never sees another's data.

Caching: Where It Lives and What Goes Wrong | System Design Concepts 11

Notes

Why cache

Caching works because of two facts. Reads vastly outnumber writes, and a small set of items gets most of the traffic: the top one percent of products might get half the views. So keep that hot set in memory, where a read takes half a millisecond instead of ten or twenty from the database.

Why caching works

reads ≫ writes (often 100:1)
a few items get most traffic:
top 1% of products ≈ half
of all views
memory read ~0.5 ms
database query ~5–20 ms
keep the hot set in memory

Layers

Caches live at every layer: the browser, the CDN, the reverse proxy, the application's own memory, a shared cache like Redis, and even inside the database. The closer to the user, the faster, but the harder it is to change when the data changes.

Where caches live

browser cache (Cache-Control)
CDN edge (episode 12)
reverse proxy (Nginx)
app memory (per server)
shared cache (Redis, Memcached)
database buffer pool
closer to the user = faster,
but harder to invalidate

Cache-aside

The most common pattern is cache-aside. The app checks the cache. On a hit, done. On a miss, it reads the database, stores the result in the cache with a time to live, and returns it. When the data changes, update the database and delete the cache key, so the next read loads the fresh value.

Cache-aside (lazy loading)

v = cache.get(key)
if v is None:
    v = db.query(...)
    cache.set(key, v, ttl=600)
return v

on write: update DB,
then DELETE the cache key

most common pattern

Writes

There are other write strategies. Write-through updates the cache and the database together: always fresh, slower writes. Write-back writes to the cache and flushes to the database later: very fast, but a crash can lose data. Write-around writes only to the database and lets the cache fill on the next read.

Write strategies

write-through:
write cache + DB together
always fresh, slower writes
write-back (write-behind):
write cache now, DB later
very fast, can LOSE data
write-around:
write DB only; cache fills
on the next read

Invalidation

Cached data goes stale. Give every entry a time to live as a safety net, and delete keys on writes for freshness. Memory is limited too, so the cache evicts entries: least recently used is the usual policy. In Redis you set a maximum memory and an eviction policy.

Invalidation and eviction

TTL: every entry expires
short TTL = fresher, more misses
delete on write: freshest
memory is limited → evict:
LRU: least recently used
LFU: least frequently used
set a max memory and an
eviction policy in Redis

Thundering herd

The classic failure is the thundering herd. A very hot key expires, and in that instant thousands of requests miss together and all hit the database, which falls over. Fixes: let only one request refill the key while others wait, refresh popular keys before they expire, add jitter to TTLs, or serve the stale value while refreshing.

Stampede fixes

a hot key expires →
every request misses at once →
database overloads
fixes:
single flight / lock: one
request refills, others wait
refresh early, before expiry
add random jitter to TTLs
serve stale while refreshing

Pitfalls

Other traps. Races can leave stale data behind, so keep TTLs short or put a version in the key. One celebrity item can overload a single cache node, so add a small local cache or spread copies. And if the cache restarts empty, all traffic hits the database, so plan for a cold start.

Other things that go wrong

stale data: delete-then-race
→ short TTLs, versioned keys
hot key: one celebrity item
→ local cache, key copies
cache down: all load hits DB
→ size DB for it, or degrade
caching errors / empty results
→ cache "not found" briefly

Say it like this

The interview answer: a Redis cache with cache-aside, keys like product colon id with a ten-minute TTL, deleted on update, LRU eviction, single-flight refills to protect hot keys, and a CDN for images. Then say clearly which data may be stale, and for how long.

In the interview

"How do you make reads fast?"
Redis, cache-aside
key: product:{id}, TTL 10 min
delete key on update
LRU eviction
protect hot keys: single flight
CDN for images and static files
say what may be stale, and how long