Skip to content
Episode 73 · 3,000 identical questions a second

Caching strategies

Thursday, 18:40: hotel pages send 3,000 reads a second to PostgreSQL and its CPU sits at 90%. Omar checks a sample: 95% of those reads return exactly what they returned a second ago.

A cache can make a page 40 times faster. It can also show guests a room that's already sold. Here's how to get the first without the second.

30 min intermediateCachingCache patterns: cache-aside, read-through, write-through, write-backCache invalidation and TTLsPreventing cache stampedesCache layers: browser, CDN, application, databaseCaching static assets with hashed filenames and CDNs
  1. Problem

    The hotel page reads prices and availability from PostgreSQL on every view. At 3,000 views per second the database runs at 90% CPU, and 95% of those reads return exactly what they returned a second ago.

  2. Why the simple system fails

    Adding app servers sends even more identical queries to the same database. Read replicas help but cost a full database each, and every read still pays a network round trip plus a query.

  3. Engineering concept

    Keep a copy of hot, expensive-to-compute data closer to the reader, and decide explicitly how the copy is filled, how writes update it, and how stale it may get.

  4. Technology

    Browser cache and CDN (Cache-Control headers), in-process caches, Redis or Memcached with cache-aside / read-through / write-through / write-back, TTLs, invalidation on write, request coalescing.

  5. Trade-offs

    Every cache trades freshness and simplicity for speed. You now have two copies that can disagree, a new failure mode (cold or crashed cache), and stampedes when hot keys expire.

01

The problem

2 min
Thursday, 18:40. Evening peak on hotel pages.
  1. Omar · On-call SRE
    Postgres CPU 90%. Hotel pages send 3,000 reads a second. In a sample, 95% return exactly what the same read returned a second ago.
  2. Karim · CTO
    A bigger database is €2,000 a month more. Anything cheaper?
  3. Lina · Senior engineer, your mentor
    You lead this one. I'll check your answers.
02

The big idea

4 min
Stop asking the same question
The big idea

A cache is a fast copy of slower data. Every copy needs a fill rule, a write rule and a freshness limit, and must never decide what only the truth can.

3,000 reads become 150 a second. Lina's one warning: a hotel's price can be a minute old; whether the last room is free cannot.

03

How it works

5 min
A hotel-page read with cache-aside
  1. 1
    Check the copy

    Cache-aside: the app asks Redis first for price:42:2026-12-20. A hit returns in ~1 ms. The is the share answered there: at 95%, only 5% reach the database.

  2. 2
    Miss: ask the truth

    On a miss, it queries Postgres (~40 ms).

  3. 3
    Fill the copy

    It stores the answer with a (time to live) of ~60 s, randomised ±20% so keys don't expire together.

  4. 4
    Price changes

    Invalidation: the owner's update writes Postgres first, then deletes the key. The next read refills from the truth.

  5. 5
    Safety net

    If the delete is lost, the TTL still limits staleness to about a minute.

  6. 6
    Truth at checkout

    Availability is cached 30 s for browsing only. Checkout reads availability and price from Postgres, with a row lock.

04

Watch a decision being made

4 min
Worked example

You cache hotel pages safely

Cut database load sharply without ever selling a room that's gone or showing a price that never existed. Lina checks the write path.

  1. 1Sort the data by staleness cost

    Details (weekly): a minute stale is harmless. Prices (daily): acceptable, re-checked at checkout. Availability (constant): 30 s for browsing, never trusted at checkout.

    Why: The cost of a stale read decides the TTL, and whether caching is allowed at all.

  2. 2Keys, TTLs, and an optional cache

    hotel:{id} 10 min, price:{id}:{night} 60 s, avail:{id}:{night} 30 s, all ±20%. If Redis is down, fall back to Postgres after a 20 ms timeout.

    Why: Keys match how pages read, so many guests share each copy. A Redis outage should make pages slower, not broken.

  3. 3Write path: truth first

    Update Postgres, then after commit delete the key.

    ts
    await db.query(UPDATE_PRICE, [amount, hotelId, night]);
await redis.del(`price:${hotelId}:${night}`); // after the commit

    Why: Lina: "truth first, then remove the copy. Never the other way round."

  4. 4Measure

    Hit ratio 95%, Postgres reads 3,000 → 150/s, CPU 90% → 25%, page p95 180 → 45 ms. Alert if the hit ratio drops below 85%.

    Why: A falling hit ratio warns that keys or TTLs no longer match how guests read.

The whole thing in one line: Cache the pieces guests read most, with TTLs set by how much staleness each can afford; delete after every write; keep checkout on the database. Database reads fell twentyfold.
05

In real life

3 min
Cache whole pages, or the data behind them?

Wikipedia: cache whole pages

  • Has publicly described serving most anonymous readers from caching proxies
  • An article looks the same for every logged-out reader
  • Huge savings; edits must purge the cached page

Works when the same page is shown to everyone.

Facebook: cache the pieces

  • Its published memcache paper describes a huge look-aside cache of data, not pages
  • Every page is personal, assembled per user from cached pieces
  • Needs careful invalidation and stampede protection

Works when every page differs but is built from shared data.

06

What goes wrong

3 min
No stampede protection on hot keys

Every concurrent request refills the expired key at once.

Instead

Coalesce misses per key, serve stale while one request refreshes, jitter TTLs.

Trusting cached availability at checkout

"2 rooms left" after the last one sold: two guests book it.

Instead

Browse from the cache; check and lock in the database at checkout.

Caching before fixing the query

A cold start or an outage brings the slow query back at full force.

Instead

Fix the query first; cache what's still expensive and repeated.

07

Trade-offs

3 min
Freshness ruleGainsCosts
Short TTL onlySimpleStale for up to the whole TTL
Delete on write + TTLUsually fresh right after a changeEvery write path must invalidate
Don't cacheAlways freshFull database load, slower pages
Monday, 10:00. How long to cache availability?
  1. Maya · Product manager
    Guests hate seeing "2 rooms left", clicking, and finding it sold. Don't cache availability at all.
  2. Omar · On-call SRE
    It's half of the reads. Uncached, Postgres goes back to 60% CPU at peak.
  3. Omar · On-call SRE
    And checkout re-checks anyway, so nobody is double-booked.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • What may be cached and for how long: details and prices 60 s (±20%), availability 30 s for browsing only, nothing personal in a shared cache.
  • What is never trusted from the cache: availability and price at checkout.
  • How hot keys are protected, and what happens if Redis is empty or down: slower, never broken.
Tell Claude Code
Add cache-aside caching in Redis for hotel page reads (details plus prices for the chosen nights), which today send ~3,000 reads/s to PostgreSQL at 90% CPU. TTL 60 s with ±20% jitter; delete the key after any update commits. Only one request per key may reload on a miss; others wait or get the stale value. Cache availability 30 s for browsing only; checkout keeps reading the database under its lock. Nothing guest-specific in the cache. Plan first, then show hit ratio, database reads/s and page p95 in a staging load test, including one with Redis off.
Check before you accept
  • The checkout path never reads prices or availability from the cache.
  • Write order: database commit first, then DEL. Ask Claude to point at the lines.
  • A test proves 800 simultaneous misses on one key run only one query; with Redis off, slower but no errors.
Red flags
  • It reads the cached price or availability during checkout.
  • It updates the cache before the database write commits.
  • It caches personalised responses in a shared cache or CDN.
09

Check yourself

2 min
Q1/3Reasoning
StayHub's hotel pages get 3,000 reads/s at peak. With a cache hit ratio of 95%, how many reads per second reach the database?
reads/s
How confident are you?

Next episode: booking confirmation takes 1.3 seconds, Maya wants 400 ms, and five teams each own a piece of it. You lead.

Finished?

Mark it complete to update your skill map.

Next: Latency budgets and the critical path
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Keeping the copy honest

You lead the design. Lina only checks the write path and adds the warning about availability.

Worked example

You cache hotel pages safely

Hotel pages send 3,000 reads/s to Postgres (CPU 90%), 95% of them repeats. Cut database load sharply without ever selling a room that's gone or showing a price that never existed.

  1. 1Sort the data by staleness cost

    Hotel details (changes weekly): a minute stale is harmless. Prices (change daily): a minute stale is acceptable, re-checked at checkout. Availability (changes constantly): 30 s stale for browsing, never trusted at checkout.

    Why: The cost of a stale read decides the TTL, and whether caching is allowed at all.

  2. 2Pick keys and TTLs

    hotel:{id} (TTL 10 min), price:{id}:{night} (60 s), avail:{id}:{night} (30 s), each randomised ±20%.

    Why: Key per hotel and night matches how pages read the data, so many guests share each copy. Jitter stops keys written together from expiring together.

  3. 3Write the read path

    Cache-aside: Redis first; on a miss, query Postgres, store with the TTL, return. If Redis is down, fall back to Postgres with a short timeout on Redis (20 ms).

    Why: The cache must be optional: a Redis outage should make pages slower, not broken.

  4. 4Write the write path

    Owner updates price → update Postgres → after commit, DEL price:{id}:{night}.

    ts
    await db.query(UPDATE_PRICE, [amount, hotelId, night]);
await redis.del(`price:${hotelId}:${night}`); // after the commit

    Why: Lina: "truth first, then remove the copy. Never the other way round."

  5. 5Protect the truth at checkout

    Checkout reads availability and price from Postgres inside the booking transaction, with a row lock. The cached value is only used to show the page.

    Why: A cache can say "2 rooms left" for 30 s after the last one sells. Only the database decides a booking.

  6. 6Measure

    Hit ratio 95%, Postgres reads 3,000 → 150/s, CPU 90% → 25%, page p95 180 → 45 ms. Alert if the hit ratio drops below 85%.

    Why: A falling hit ratio is the early warning that keys or TTLs no longer match how guests read.

The whole thing in one line: You cached the pieces guests read most, with TTLs set by how much staleness each can afford, deleted copies after every database write, kept checkout on the database, and cut database reads twentyfold.
PatternRead pathWrite pathStalenessMain risk
Cache-asideApp: cache, then DB on miss, then fillDB, then delete keyUp to TTL if invalidation fails or racesStampede on hot key expiry; app owns the logic
Read-throughCache library loads from DB on missDB, then invalidateSame as cache-asideCache layer becomes a critical dependency
Write-throughCache (always warm for written keys)Cache and DB synchronouslyLow for data written through itSlower writes; caches data nobody reads
Write-backCacheCache only; flushed to DB laterDB lags behindData loss if the cache dies before flushing
Cache-aside reads with delete-on-write is the common default.
http
# Fingerprinted asset: content never changes at this URL
Cache-Control: public, max-age=31536000, immutable
# Anonymous city landing page: CDN may cache 60 s, serve stale 5 min while refreshing
Cache-Control: public, s-maxage=60, stale-while-revalidate=300
# Personal pages: never stored by shared caches
Cache-Control: private, no-store
HTTP caching headers for the layers in front of the app.
11

Interactive simulation

12

Visual diagram

Click a component

Cache layers in a hotel site, from the guest's browser to the database. Walk through a read, a write, and a stampede.

13

Analogy

  • Sticky notes at a hotel deskHead office system, slow to queryCachingThe database (source of truth)
  • Sticky notes at a hotel deskCheck the notes, else call and write oneCachingCache-aside
  • Sticky notes at a hotel desk"Valid until 18:00" on each noteCachingA TTL (expiry time)
  • Sticky notes at a hotel deskHead office: "tear up room 12's note"CachingInvalidation
  • Sticky notes at a hotel deskUpdate the note, fax head office tonightCachingWrite-back: fast, can lose data
  • Sticky notes at a hotel desk30 receptionists call head office at onceCachingA cache stampede

Where the analogy breaks: a desk has a handful of notes. Real caches hold millions of entries at several layers (browser, CDN, app, database), each with its own expiry.

14

Check your understanding

Q1/3Decision
Match each data type to the most fitting strategy:
Hotel descriptions, read thousands of times per edit
Page-view counters, loss of a few seconds is fine
User settings, read right after saving
Fingerprinted JS bundle
How confident are you?
15

Practice ladder

Step 1 · Trace

Trace a price change

Scenario Warm-up

Cache-aside is live with TTL 60 s (±20%) and delete-on-write. At 10:00:00 the key price:42:2026-12-20 holds €120 (written at 09:59:30).

  • 10:00:10: the owner changes the price to €95. The update commits; the delete succeeds.
  • 10:00:11: guest A opens the page.
  • 10:05:00: the owner changes it to €99. The update commits; the delete fails (network blip).
  • 10:05:05: guest B opens the page. (The page is busy, so the key is refilled about once a minute; the last refill before the second update was at 10:04:40.)

Trace what each guest sees, and the latest time the wrong price can still be shown.

Step 2 · Complete

Finish the coalescing function

Code Practice

Lina's function to stop stampedes inside one pod has two gaps. Concurrent misses for the same key should share one database call.

const inFlight = new Map<string, Promise<Price>>();

export function getPriceCoalesced(hotelId: string, night: string): Promise<Price> {
  const key = `price:${hotelId}:${night}`;
  const pending = inFlight.get(key);
  if (pending) ___1___;
  const p = getHotelPrice(hotelId, night)   // cache-aside read
    .finally(() => ___2___);
  inFlight.set(key, p);
  return p;
}

Fill the gaps, then explain what this does not protect against when 20 pods run.

Step 3 · Solve

Cache the search results?

Design Practice

Search queries like "Paris, 20–22 Dec, 2 guests" cost ~180 ms in the database. Omar's data for one day:

  • 4 million searches; the top 2,000 distinct searches make up 60% of them; the rest are almost all unique.
  • Results include prices and a "rooms left" badge.
  • Prices change ~once a day; availability changes many times an hour for popular hotels.

Design a cache for search: what to cache, the key, the TTL, how to keep it safe, and the expected database savings.

Step 4 · Decide

Write-back for loyalty points?

Decision Stretch

The loyalty team wants to make "points earned" updates faster. Each booking adds points; guests check their balance often. Proposals:

  • A (write-back): update the points balance only in Redis; flush to Postgres every 5 minutes. Writes take ~1 ms instead of ~15 ms.
  • B (cache-aside): write Postgres (~15 ms), delete the cached balance; reads use the cache.

Points can be spent on bookings (1 point = €0.01). About 30,000 point updates a day.

Write a short decision: "We chose X over Y because…, accepting the risk of…".

16

Explain it yourself

Explain to a teammate how you would cache hotel prices, and what can go wrong.

0 words