# Overview A cache is a temporary storage area used for quick look-ups within a system. A system may want quick look-ups to improve performance or to reduce costs. For example, a website may use a cache to store large images so a web page can load quickly, thus improving performance. Or a data processing application that uses [[DynamoDB]] may cache key data elements to reduce the number of queries run against the database, which charges for each read or write on a table. That sounds amazing! Why not always use a cache? Well, a cache acts as quick, temporary storage which means it cannot scale without incurring large costs. As such, there are various details which must be considered to design an effective cache. ## General Use Cases - Save aggregated metrics - Reduce number of DB queries - Speed up expensive queries - Popular static content # Key Considerations While a cache does improve performance, it also introduces complexity to design and maintain ## Designing an Effective Cache ### Cache Coherence | Type | Topic | Description | Pros | Cons | Use Cases | | -------------- | ----------------------- | --------------------------------- | ----------------------------- | ----------------------------- | ---------------------------------- | | Write Strategy | [[Write-through Cache]] | ![[Write-through Cache#Overview]] | ![[Write-through Cache#Pros]] | ![[Write-through Cache#Cons]] | ![[Write-through Cache#Use Cases]] | | Write Strategy | [[Write-back Cache]] | ![[Write-back Cache#Overview]] | ![[Write-back Cache#Pros]] | ![[Write-back Cache#Cons]] | ![[Write-back Cache#Use Cases]] | | Write Strategy | [[Write-around Cache]] | ![[Write-around Cache#Overview]] | ![[Write-around Cache#Pros]] | ![[Write-around Cache#Cons]] | ![[Write-around Cache#Use Cases]] | | Read Strategy | [[Cache-aside]] | ![[Cache-aside#Overview]] | ![[Cache-aside#Pros]] | ![[Cache-aside#Cons]] | ![[Cache-aside#Use Cases]] | | Read Strategy | [[Read-through Cache]] | ![[Read-through Cache#Overview]] | ![[Read-through Cache#Pros]] | ![[Read-through Cache#Cons]] | | ### Eviction Policy Caches are expensive and there goal is to be performant. To address both of these areas, there needs to be an eviction process to keep the size of the cache manageable. There are many different approaches to select from shown in the table below. | Topic | Description | Pros | Cons | Use Cases | | ------------------ | ---------------------------- | ------------------------ | ------------------------ | ----------------------------- | | [[FIFO - Caching]] | ![[FIFO - Caching#Overview]] | ![[FIFO - Caching#Pros]] | ![[FIFO - Caching#Cons]] | ![[FIFO - Caching#Use Cases]] | | [[LRU - Caching]] | ![[LRU - Caching#Overview]] | ![[LRU - Caching#Pros]] | ![[LRU - Caching#Cons]] | ![[LRU - Caching#Use Cases]] | | [[LFU - Caching]] | ![[LFU - Caching#Overview]] | ![[LRU - Caching#Pros]] | ![[LRU - Caching#Cons]] | ![[LRU - Caching#Use Cases]] | ### Cache Invalidation In many systems, updates to the data are made directly to the database. This means that data may be updated, but the cache still holds an old version of the data. A cache invalidate strategy is an approach to invalidate stale data in the cache and get updates from the database. Approaches for cache invalidation include... #flashcard 1. **Time-based expiration (TTL)** - Set a fixed lifetime for cached entries. Simple to implement but means serving potentially stale data until expiration. Works well for data with predictable update patterns. 2. **Write-through invalidation** - Update or delete cache entries immediately when writing to the database. Ensures consistency but adds latency to write operations and requires careful error handling. 3. **Write-behind invalidation** - Queue invalidation events to process asynchronously. Reduces write latency but introduces a window where stale data might be served. 4. **Tagged invalidation** - Associate cache entries with tags (e.g., "user:123:posts"). Invalidate all entries with a specific tag when related data changes. Powerful for complex dependencies but requires maintaining tag relationships. 5. **Versioned keys** - Include version numbers in cache keys. Increment the version on updates, naturally invalidating old cache entries. Simple and reliable but requires version tracking. <!--ID: 1753886610163--> #### How do you handle cache invalidation when data updates need to be immediately visible? #flashcard Use cache versioning requires two cache reads per request. Instead of invalidating caches, you change the cache key when data updates: 1. **Read version number** from cache (or database if cache miss) 2. **Construct versioned key** using that version number 3. **Read actual data** from cache using the versioned key 4. **On cache miss**, fetch from database and cache with versioned key This approach trades one additional cache read for elimination of invalidation complexity. Old cache entries become naturally unreachable while new requests automatically use fresh version numbers. No race conditions, no partial invalidation failures, but every request now requires checking the version first. - When versioning isn't practical - perhaps you're caching search results or aggregated data - you need explicit invalidation strategies. Write-through caching updates the cache immediately when you update the database, ensuring consistency at the cost of write latency. - Another approach uses a deleted items cache, which is a much smaller working set of items that have been deleted, hidden, or changed. Instead of invalidating all feeds containing a deleted post, you maintain a cache of recently deleted post IDs. When serving feeds, you first check this small cache and filter out any matches. This lets you serve mostly-correct cached data immediately while taking time to properly invalidate the larger, more complex cached structures in the background. The deleted items cache can be small (just recent deletions) and fast to query, making this pattern very efficient for content moderation and privacy changes. - For global systems with CDN caching, invalidation becomes even more complex. You're not just clearing one cache but potentially hundreds of edge locations worldwide. CDN APIs help, but propagation takes time. Critical updates might use cache headers to prevent CDN caching entirely, trading performance for consistency. Less critical data can use shorter TTLs at the edge while maintaining longer TTLs in your application cache. <!--ID: 1753886610166--> # Implementation Details ## Cache Locations A cache can be implemented in most layers of an architecture: - [[Browser Cache]] - [[CDN Cache]] - [[API Gateway Cache]] - On the [[Computer Server]]: - [[L1 Cache]] - [[L2 Cache]] - [[L3 Cache]] - [[Page Cache]] - On the [[Databases]]: - [[Buffer Cache]] - [[Query Cache]] - [[Session Cache]] - [[Metadata Cache]] ## Cache Size Limitations ![[Back of the Envelope Estimation#Caching]] # Useful Links ## Products - [[Redis]] - [[Memcached]] # Related Topics - [[Content Delivery Network (CDN)]] --- When adding a cache to a system design, you'll want to consider the... #flashcard - Key and value fields used within the cache - Cache Sizing and Hit Ratios - Caching Coherence Strategy (cache-aside, write-through, etc.) - Cache Eviction Policy - Cache Invalidation - TTL <!--ID: 1751507776560--> ---