Distributed Caching: Policies, Eviction & Invalidation

why an application-level cache, beyond the CDN

A CDN caches static assets at the network edge. A distributed cache solves a different problem: repeated, expensive computation or database reads inside the application itself. When PlateRoute's homepage needs "top-rated restaurants near this ZIP code," recomputing that from the database on every single request wastes work that's identical for thousands of nearby users within the same few minutes.

when to cache, and when not to

Good fit for caching	Poor fit for caching
Expensive-to-compute or expensive-to-fetch data requested repeatedly (top restaurants, a user's session)	Data that's write-heavy and rarely re-read (an audit log)
Data that tolerates some staleness (restaurant ratings, view counts)	Data with strict consistency requirements (a live account balance mid-transaction)
Data requested often enough that a cache hit is likely before it expires	Data requested once and never again (a one-off report)

writing policies

Policy	Behavior	Trade-off
Write-through	Every write goes to the cache and the database together (or the cache write triggers the database write).	Strong cache/database consistency, but higher write latency.
Write-back	Writes land in the cache first and are asynchronously flushed to the database later.	Very low write latency, but risks losing recent writes if the cache fails before flushing, and risks stale reads elsewhere.
Write-around	Writes go straight to the database, bypassing the cache; the cache is only populated on a subsequent read (cache miss).	Avoids filling the cache with data that might never be re-read, but the first read after a write is always a slow cache miss.

eviction: what to remove when the cache is full

A cache is finite RAM, so something has to be evicted once it's full. The dominant policy is LRU (least recently used) — evict whatever hasn't been touched in the longest time, on the assumption that recent access predicts near-future access. Variants exist for specific access patterns: LFU (least frequently used) favors items with high total access count over merely-recent ones, and a combined LFRU tries to balance both signals.

Alongside eviction, most cache entries also carry a TTL (time-to-live), so stale data gets removed even if it would otherwise stay "hot" enough to avoid LRU eviction. TTL expiry can be checked actively (a background sweep) or passively (checked only when the entry is next accessed) — passive is cheaper but means a stale, unused entry can linger in memory until something happens to touch it.

scaling a cache beyond one machine

A single cache server is both a capacity ceiling and a single point of failure. Two decisions shape how a cache cluster is built:

Sharding — using consistent hashing, each key is deterministically mapped to one shard, so a cache client always knows which node to ask without a lookup step. Placement — dedicated cache servers (a standalone caching tier, usable by multiple services) versus co-located caches (embedded on the same host as the application, avoiding a network hop but coupling the cache's lifecycle to the application's).

For availability, each shard is typically backed by a primary plus one or more replicas, following the same replication trade-offs covered for databases — synchronous replication within a shard for consistency, at some added write latency.

Memcached vs. Redis, briefly

	Memcached	Redis
Data model	Simple key-value strings	Rich data structures — strings, hashes, lists, sets, sorted sets
Persistence	Purely in-memory, no disk persistence	Optional disk persistence (snapshotting or an append-only log)
Typical use	Straightforward object caching	Caching plus lightweight structures like leaderboards, rate-limit counters, or pub-sub

Neither is universally "better" — Memcached's simplicity keeps it fast and predictable for pure caching; Redis's richer feature set makes it a common default when the same store is also asked to do double duty as, say, a rate limiter or a lightweight queue.

reference

Redis Docs — Key Eviction