Editorial note: This is a vendor-neutral educational draft. It is not a disclosure of a specific production system.
“Add a cache” is often treated as an implementation detail. In a distributed system, it is more consequential: the cache answers questions that a source of truth used to answer. That means it inherits an API contract, even if nobody writes one down.
Decide who owns the cached answer #
The first useful question is not TTL. It is ownership. Is the cache local to one process, shared by a service, or an explicit data product another service may consume? Each choice determines who can invalidate, who can observe freshness and who carries the cost of a miss.
Shared caches are powerful because they collapse repeated work. They also turn an operational dependency into part of the request path. The system needs to know what happens when it is slow, empty or unavailable.
Make freshness legible #
A single TTL encodes a business decision. Is a value allowed to be five minutes old? Can the user see an old value during a source outage? Which updates must invalidate immediately? The answer varies by data type; treating every object with the same expiration policy creates accidental correctness rules.
Cache contract: owner = catalog service; freshness = 60 seconds; on miss = bounded source read; on source outage = stale for 10 minutesDesign cache failure as a normal state #
A cache can fail open, fail closed or fail stale. None is automatically correct. A read path that falls back to a datastore on every cache failure might protect correctness while creating a database overload. A stale response might protect availability while violating an expectation the product cannot tolerate.
- Bound stampedes. Coalesce misses, use request-level single-flight or refresh before expiry for hot keys.
- Separate value from availability. A cache miss and a cache outage are different signals with different fallback paths.
- Instrument freshness and misses. Hit rate alone does not show whether the cache is useful or safe.
- Test invalidation as a product scenario. A correct update path includes what every reader sees next.
Use the cache to remove work, not hide it #
The strongest cache improvements reduce clearly identified repeated work: unnecessary query volume, duplicate computation or expensive fan-out. They also make cost more legible. If a cache improves p99 latency but introduces an unbounded hot-key reload pattern, the bill may simply move elsewhere.
The operating rule #
Before choosing a key or TTL, write the answer the cache is permitted to give and what it must do when it cannot give one. Then the cache becomes a dependable boundary, not an invisible layer that only works while every dependency is healthy.