Skip to content
Episode 76 · The pods we thought were stateless

Stateless services

Monday, 9:00: after Sunday night's scale-down, 1,900 guests lost the room they were holding mid-checkout, and at 08:00 this morning 4,300 guests got the same pre-arrival email eleven times.

If any server can handle any request, you can add, remove or lose servers freely. Here's how to get there.

30 min intermediateStateless serversMoving session and state out of the serverSticky sessions and their trade-offsDisposable, interchangeable processes (12-factor)IdempotencyServer sessions vs self-contained tokens (JWT)
  1. Problem

    The booking API runs on one server and keeps logged-in sessions, shopping carts and uploaded room photos in memory and on local disk. Now it needs three instances behind a load balancer.

  2. Why the simple system fails

    Guests get logged out at random, carts vanish, photos 404, and every deploy or crash wipes all of it, because each request may land on an instance that never saw that guest.

  3. Engineering concept

    Make instances interchangeable: keep no data that must outlive one request inside the process. Push sessions, files, caches and jobs to shared backing services, and make handlers safe to retry on any instance.

  4. Technology

    Redis session stores or signed tokens (JWT), object storage for files, a shared cache, queues for background work, idempotency keys, graceful shutdown, 12-factor config.

  5. Trade-offs

    Every externalised piece of state adds a network hop and a dependency that must be available and scaled. Sticky sessions avoid the work but give up even load and resilience. Signed tokens avoid the store but make revocation hard.

01

The problem

2 min
Monday, 9:00. The morning after the TV ad.
  1. Maya · Product manager
    Sunday after 22:00, about 1,900 guests saw "your hold expired" at Pay. About 140 of those rooms went to someone else.
  2. Omar · On-call SRE
    And at 08:00, 4,300 guests got their pre-arrival email 11 times. We run 11 pods on Monday mornings.
  3. Lina · Senior engineer, your mentor
    We called booking-api stateless. You lead this one.
02

The big idea

4 min
Where the holds went
The big idea

Keep nothing in a pod that a later request needs. Then any pod can serve anyone, and pods can be added, removed or killed without a guest noticing.

Nobody noticed for months because pods lived for days. Autoscaling made them come and go hourly, and turned a rare glitch into 1,900 failures.

03

How it works

5 min
From trapped state to shared state
  1. 1
    Inventory

    Search the code, not the dashboards: module-level Maps, setInterval, local files. Found: a holds Map, an 08:00 email timer, a recentBookings Map that catches retries.

  2. 2
    Holds to Redis

    Externalised state lives in shared services. SET hold:{room}:{night} {guest} NX EX 600: one hold per room and night, visible to every pod, expiring after 10 minutes.

  3. 3
    Jobs run once

    The timer becomes one Kubernetes at 08:00 that enqueues one email per guest; workers send them and mark each as sent.

  4. 4
    Retries checked in the database

    An is a unique id sent with a request, so a retry is recognised. A unique constraint in Postgres, inside the booking transaction.

  5. 5
    The truth stays in Postgres

    The booking still locks the room row. The Redis hold is a courtesy to the guest; the database row decides.

  6. 6
    Prove it

    Load test, kill pods at random, scale 32 → 11. Lost holds, duplicate emails, duplicate bookings: all zero. Now the pods are .

04

Watch a decision being made

4 min
Worked example

You make booking-api truly stateless

Move holds, the 08:00 email and retry de-duplication out of pods, so scale-downs, deploys and crashes never affect a guest. Lina comments twice.

  1. 1Move holds to Redis

    Create a hold with NX (only if free) and EX 600 (expires alone). At booking, check it belongs to this guest.

    ts
    const ok = await redis.set(`hold:${roomId}:${night}`, guestId, "NX", "EX", 600);
if (!ok) throw new RoomHeldError();

    Why: One hold per room and night across all pods, and nothing has to clean up.

  2. 2Keep the decision in the database

    The booking transaction still runs SELECT … FOR UPDATE on the room row before inserting.

    Why: Lina: "if Redis restarts, the database must still refuse a double booking."

  3. 3Run the schedule once, make writes idempotent

    Delete the timer; one CronJob enqueues. The app sends an Idempotency-Key; the transaction inserts it with a unique constraint and returns the existing booking on conflict.

    Why: However many pods run, the schedule runs once; retries land on any pod, so only shared storage can recognise them.

  4. 4Prove pods are disposable

    Sunday-peak load test while killing random pods and scaling 32 → 11. Count losses and duplicates.

    Why: Lina: "stateless is a property you test, not a word you write in the README."

The whole thing in one line: Found the hidden state, moved it to shared services with the database as final judge, and tested that any pod can vanish without a guest noticing.
05

In real life

3 min
Stateless by rule, or stateful by design?

The Twelve-Factor App (Heroku): stateless processes

  • A widely used methodology published by Heroku's engineers
  • Processes share nothing; state lives in backing services
  • Any process can be killed or added at any time

Fits request/response web apps that must scale and deploy freely.

Discord: live state in specific processes

  • Has publicly described keeping each server's live state in memory in a dedicated process
  • Users are routed to the process that owns their server
  • Fast fan-out; needs routing and careful handover

Fits real-time systems where hot state is too chatty to fetch per event.

06

What goes wrong

3 min
Running cron inside every instance

N instances run the job N times: duplicate emails, charges or reports.

Instead

One scheduler, or a queue with a lock; record what was done.

In-process caches that affect correctness

A promo code's usage limit counted per pod can be used N times.

Instead

Counters and decisions in shared storage.

Writing uploads to local disk

Files exist on one instance and vanish when it's replaced.

Instead

Object storage, served via a CDN.

07

Trade-offs

3 min
Sessions inGainsCosts
Replicated RedisInstant logout; easy to change session data~1 ms per request; a dependency to run
Signed tokens ()No store, no extra callHard to revoke; bigger cookies
Wednesday, 15:00. Redis is now on every checkout's path.
  1. Karim · CTO
    Use signed tokens: the session travels in the cookie. No store, no Redis call.
  2. Omar · On-call SRE
    And when a guest's phone is stolen, how do we log that token out?
  3. Omar · On-call SRE
    Short expiry means refresh tokens, stored somewhere we can revoke. A store again, plus token logic.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • Where each piece of state lives: holds in (600 s), bookings and idempotency keys in PostgreSQL, scheduled work in one CronJob.
  • That sticky sessions are not the fix: they hid the problem until the scale-down lost ~1,900 holds.
  • That holds are checked inside the booking transaction, so an expired hold can't be paid for and no room sells twice.
Tell Claude Code
booking-api pods must be disposable: the autoscaler goes from 32 to 11 pods on Sunday night. Find every piece of state kept inside a pod (in-memory maps, setInterval timers, local files, module-level caches) and list them with file and line; don't change anything yet. Then propose: holds to Redis with a 600 s expiry, checked in the booking transaction; the 08:00 email to one CronJob that enqueues work; retry de-duplication to an idempotency key with a unique constraint in Postgres. No sticky sessions. Add a test sending a retried POST /bookings to two instances, expecting one booking.
Check before you accept
  • Claude's list of hidden state covers setInterval, new Map and local disk writes.
  • A unique constraint on the idempotency key in the migration; a code-only "if exists, skip" races between pods.
  • Kill a pod mid-booking: the booking still completes exactly once.
Red flags
  • It turns on cookie affinity and calls the service "stateless now".
  • It adds a lock so "only pod 0 sends emails". Pods have no stable number.
  • It stores holds in Redis but never checks them in the booking transaction.
09

Check yourself

2 min
Q1/3Debugging
After adding a second API instance behind a round-robin load balancer, about half of page loads show guests as logged out. What is the most likely cause?
How confident are you?

Next episode: PostgreSQL is at 80% CPU, 94% of its queries are reads, and the disk will be full in eight months.

Finished?

Mark it complete to update your skill map.

Next: Replication and sharding
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Retries that land elsewhere

You lead the refactor; Lina's two comments are in the "why" lines.

Worked example

You make booking-api truly stateless

Room holds, the 08:00 email and retry de-duplication live inside pods. Move them out so that scaling from 32 to 11 pods, deploys and crashes never affect a guest.

  1. 1Inventory the trapped state

    Search for module-level Map/Set, setInterval/setTimeout jobs, local file writes and in-memory caches that affect decisions. Found: holds, recentBookings, the 08:00 timer.

    Why: You can't move state you haven't found. Dashboards won't show it; the code will.

  2. 2Move holds to Redis

    Create a hold with SET hold:{room}:{night} {guestId} NX EX 600. Any pod reads it. At booking, check the hold belongs to this guest.

    ts
    const ok = await redis.set(`hold:${roomId}:${night}`, guestId, "NX", "EX", 600);
if (!ok) throw new RoomHeldError();

    Why: NX gives one hold per room and night across all pods; EX makes it expire on its own, so nothing has to clean up.

  3. 3Keep the truth in the database

    The booking transaction still locks the room row in Postgres (SELECT … FOR UPDATE) before inserting the booking.

    Why: Lina: "Redis holds are a courtesy to the guest. If Redis restarts, the database must still refuse a double booking."

  4. 4Run scheduled work once

    Delete the timer. A Kubernetes CronJob at 08:00 enqueues one message per guest; workers send emails and mark each as sent.

    Why: However many API pods run, the schedule runs once, and "sent" is recorded so a retry doesn't resend.

  5. 5Make writes idempotent

    The app sends an Idempotency-Key header; the booking transaction inserts it into a table with a unique constraint and returns the existing booking on conflict.

    Why: Retries land on any pod; only shared storage can recognise them.

  6. 6Prove pods are disposable

    Load test at Sunday-peak traffic while killing random pods and scaling 32 → 11. Count lost holds, duplicate emails, duplicate bookings: all zero.

    Why: Lina: "stateless is a property you test, not a word you write in the README."

The whole thing in one line: You found the state hiding in pods, moved holds to Redis with an expiry, kept the database as the final judge, ran the schedule once through a queue, and made retries idempotent, so pods can be added, removed or killed without any guest noticing.
StateWhere it wasWhere it lives nowWhy there
Login sessionsRedis (since the Docker module)Redis, replicatedRead on every request; small; expires
Room holdsMap in each podRedis with TTL 600 sShort-lived, shared, expiring by itself
Scheduled emailsTimer in each podCronJob → queue → workersRuns once; retries without duplicates
Retry de-duplicationMap in each podPostgres, unique keyMust be correct across pods and restarts
PhotosLocal disk (long ago)Object storage + CDNLarge files, shared, durable
11

Interactive simulation

12

Visual diagram

Click a component

A stateless booking API: the instances hold nothing that outlives a request. Walk through where each piece of state went.

13

Analogy

  • A hotel front deskEach receptionist's personal notebookA web serviceData kept inside one server
  • A hotel front deskWristband: "always go to desk 2"A web serviceSticky sessions
  • A hotel front deskEveryone uses the shared booking systemA web serviceA shared session store and database
  • A hotel front deskAny free desk helps any guestA web serviceAny server handles any request
  • A hotel front deskRepeating "book room 12" at another deskA web serviceA retry that must not double-book

Where the analogy breaks: the shared booking system becomes the busy part now. Moving data out of the servers doesn't make it disappear. The shared stores must be scaled and kept reliable too.

14

Check your understanding

Q1/3Reasoning
Sticky sessions make a service stateless.
How confident are you?
15

Practice ladder

Step 1 · Trace

Trace a hold across pods

Scenario Warm-up

After the fix: holds in Redis (SET hold:{room}:{night} {guest} NX EX 600), the booking transaction locks the room row in Postgres, idempotency keys in Postgres.

At 21:58 Aiko holds room 812 for 20 Dec (request lands on pod 7). At 22:00 the autoscaler removes pod 7. At 22:03 she clicks Pay (lands on pod 2); the request times out at 5 s and the app retries (lands on pod 9). At 22:04 Kenji tries to hold the same room.

Trace each step.

Step 2 · Complete

Finish the state inventory

Design Practice

The reviews service is going from 1 instance to autoscaled pods. Your inventory found these. Fill in where each should live and why.

State found in the codeWhere it should liveWhy
const draftReviews = new Map() (guests' unsent drafts)______
setInterval(sendWeeklyDigest, 7 days)______
Uploaded review photos in /var/reviews/photos______
const badWords = loadList() (read once at start, same for all)______
let reviewsToday = 0 (rate limit: max 3 reviews per guest per day)______
Step 3 · Solve

The promo code used 9 times

Debugging Practice

Maya's campaign code "WINTER50" may be used once per guest. Finance reports a guest who used it 9 times on Sunday. The code check:

const usedCodes = new Map<string, Set<string>>(); // guestId -> codes used
export function canUse(guestId: string, code: string) {
  return !(usedCodes.get(guestId)?.has(code));
}
export function markUsed(guestId: string, code: string) {
  if (!usedCodes.has(guestId)) usedCodes.set(guestId, new Set());
  usedCodes.get(guestId)!.add(code);
}

On Sunday, booking-api ran 32 pods. Explain how 9 uses were possible and fix it so it's correct even with retries and concurrent requests.

Step 4 · Decide

Live chat: stateless or stateful?

Decision Stretch

StayHub is adding live chat between guests and hotels. Each open chat is a WebSocket connection held by one pod. Proposals:

  • A: connections live on whichever pod accepted them; messages to a guest are published through Redis pub/sub so the pod holding that guest's connection delivers them. Chat history is in Postgres.
  • B: route each hotel's chats to a specific pod that keeps the conversation state in memory (sticky by hotel), like Discord's guilds.

Facts: ~20,000 open chats at peak, most quiet; pods are scaled down nightly; deploys happen daily.

Write a short decision: "We chose X over Y because…, accepting the risk of…".

16

Explain it yourself

Explain what "stateless service" means, and how you would make an existing service stateless.

0 words