Skip to content
Episode 39 · A locked door at 23:30

Design a chat app (WhatsApp-lite)

Thursday, 08:40: a guest landed at 23:30 and found her hotel locked, because the hotel's reply to her email arrived 9 hours later. Maya wants guest↔hotel chat where messages arrive in under a second, even after a phone was offline.

Chat looks like "send text, show text". Underneath, the server must push messages to people who never asked, even when they're offline.

30 min advancedReal-time delivery & fan-outSeparating read and write pathsStateless serversSynchronous vs asynchronous communicationIdempotencyAt-least-once delivery
  1. Problem

    Deliver a message from Ana's phone to Bruno's in under a second, even if Bruno is offline, on a train, or has three devices.

  2. Why the simple system fails

    HTTP polling wastes battery and servers; one chat server can't hold 10M connections; a crash between 'stored' and 'delivered' loses or duplicates messages.

  3. Engineering concept

    Persistent connections on a stateless-ish gateway tier, a routing layer that knows which gateway holds whom, durable per-conversation storage with sequence numbers, and acks at each hop.

  4. Technology

    WebSockets, a connection registry (Redis), a message store partitioned by conversation, a queue/log for fan-out, APNs/FCM push, object storage + CDN for media.

  5. Trade-offs

    Fan-out on write vs on read for groups, at-least-once delivery + dedup vs complexity, presence accuracy vs heartbeat cost, end-to-end encryption vs server-side features.

01

The problem

2 min
Thursday, 08:40. Maya, with a one-star review.
  1. Maya · Product manager
    Guest landed at 23:30, emailed about late check-in. The hotel replied at 08:30. She slept in a taxi.
  2. Maya · Product manager
    I want chat: guest and front desk, messages in under a second.
  3. Lina · Senior engineer, your mentor
    You lead this one. How does the message reach the front-desk tablet?
02

The big idea

4 min
Who holds the line to the hotel?
The big idea

In chat, messages are easy and connections are the load: hold them on their own tier, store before you tick, and treat offline as a later resync.

Each online device holds an open line to a switchboard operator; a directory says which operator holds whose line. If someone has hung up, the message goes to the post office and they get a postcard: "you have mail". Every message carries a tracking number.

03

How it works

5 min

A is a connection that stays open so both sides can send any time. Put them on our app servers and every deploy, three times a day, drops them all. So a small gateway tier, servers whose only job is holding connections, sits in front; chat rules stay in the app.

Guest to front desk
  1. 1
    Send

    The guest's app sends {clientMsgId, conversationId, text} over its WebSocket to gateway 1.

  2. 2
    Store, then ✓

    The chat service checks the guest belongs to this stay, gives the message a sequence number (seq = 7, the server's order for the conversation), stores it, then acks: one tick.

  3. 3
    Find the tablet

    A Redis registry says the hotel's two devices are on gateway 2. Heartbeats every 30 s keep it fresh.

  4. 4
    Deliver

    Gateway 2 pushes the message down the tablet's socket. The tablet's delivery receipt gives the guest two ticks.

  5. 5
    Offline?

    No live socket: a . On reconnect, the app asks for messages after its last seq.

About 1.2 messages a second at the 10× plan: PostgreSQL stores them easily. The hard part is 50,000 open connections and what happens when they move or drop.

04

Watch a decision being made

4 min
Worked example

You design StayHub chat; Lina checks

Guest↔hotel chat for the 10× plan: 18,000 hotels with ~2 devices each, ticks, under a second when online, nothing lost when one side is offline.

  1. 1Estimate messages and connections

    Messages ≈ 1.2/s; sockets ≈ 50,000; ~80 GB over 2 years.

    text
    messages:    120,000/day ÷ 10⁵ s ≈ 1.2/s → PostgreSQL is plenty
sockets:     36,000 + 14,000 ≈ 50,000     → 2 gateways at 25k, run 3
polling alt: 50,000 ÷ 10 s = 5,000 req/s  > 1,600 our app tier serves

    Why: The message rate is trivial; connections are the real load. The effort goes into gateways, not the store.

  2. 2Give connections their own tier, and a registry

    3 gateways, deployed separately and drained slowly. Each writes device → gateway to Redis with a 90 s TTL, refreshed every 30 s; delivery publishes to each gateway's channel ().

    Why: App deploys several times a day would drop every socket. A TTL clears a crashed gateway's entries within 90 s.

  3. 3Store first, then tick, then deliver

    Insert with seq and a client_msg_id unique per conversation, ack one tick, then deliver. Delivery is ; devices drop a seq they already have.

    Why: Acking before storing turns a gateway crash into a message the guest believes was sent. The unique id makes resends harmless.

  4. 4Offline and reconnects

    Push when no socket. On reconnect, after a random 1–30 s delay, fetch seq > lastSeq. Photos go to object storage; messages carry the reference.

    Why: Offline becomes a normal resync, and spreading reconnects prevents a herd of thousands at once.

The whole thing in one line: 1.2 messages a second is trivial, 50,000 sockets aren't: a gateway tier with a Redis registry, store before every tick, harmless resends, and a jittered resync after the last sequence number.
05

In real life

3 min
Keep the history, or only pass it on?

WhatsApp: pass it on, can't read it

  • Publicly uses end-to-end encryption by default
  • Has described keeping messages on servers only until delivered

Fits private conversations where privacy comes first.

Slack: keep everything, searchable

  • Keeps a workspace's history on its servers; newcomers scroll back and search
  • Admins apply retention and export rules

Fits shared work conversations a team must hand over.

06

What goes wrong

3 min
Acking when the gateway receives the frame

A crash before storing loses a message the sender sees as sent.

Instead

Ack only after it's stored with its seq.

Ordering messages by the phone's clock

Clocks are wrong and senders race: replies before questions.

Instead

A server-assigned seq per conversation.

Sending photos through the WebSocket

Multi-megabyte frames block the connection.

Instead

Object storage with a signed URL; send a reference.

07

Trade-offs

3 min
TransportFits whenYou pay
PollingA pilot; tens of seconds are fineDelay; empty requests that grow with devices
Server-Sent EventsLive feeds where the client rarely sendsOne direction; still needs routing
WebSockets + gatewaysTwo-way, sub-second, many devicesNew tier, registry, reconnect logic
Friday, 14:00. Karim has the launch date in mind.
  1. Karim · CTO
    Why not have apps ask for new messages every 10 seconds? Ship in two weeks.
  2. Lina · Senior engineer, your mentor
    For the 200-hotel pilot, fine. At 10× it's 5,000 requests a second, mostly empty. Our app tier serves 1,600.
  3. Maya · Product manager
    And for "I'm outside, the door is locked", 10 seconds feels broken.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • When the sender gets one tick: only after the message is stored. A promise you set, not the agent.
  • Where sockets live: 3 separate gateways, deployed and drained apart from the app; messages ordered by a server seq, never the phone's clock.
  • How offline devices catch up (push, then seq > lastSeq) and how reconnects spread out (random 1–30 s).
Tell Claude Code
Build guest-to-hotel chat. Plan first, wait for approval. Decided: WebSockets end on a separate gateway process (3 instances); gateways register device-to-gateway in Redis, 90 s TTL refreshed every 30 s; the chat module stores each message with a per-conversation seq and a client_msg_id unique per conversation, acks only after the insert commits, then delivers to every device. Offline: push, then fetch seq above the last one. Reconnects wait a random 1–30 s. Test sending one client_msg_id twice.
Check before you accept
  • The ack to the sender comes after the insert succeeds.
  • A unique constraint on (conversation_id, client_msg_id); a resend returns the existing message.
  • The WebSocket server isn't inside the main app, and the reconnect delay is random.
Red flags
  • It acks when the gateway receives the message.
  • Messages ordered by createdAt from the phone.
  • WebSockets on the app servers "to keep things simple".
09

Check yourself

2 min
Q1/3Debugging
Users report messages that showed one tick but never arrived. Logs show a gateway restarted at that time. What's the most likely design flaw?
How confident are you?

Next episode: the big one. Karim wants the booking engine itself redesigned for 10×, and this time Lina just hands you the marker.

Finished?

Mark it complete to update your skill map.

Next: Design a hotel booking engine
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Never lose it, never show it twice
Worked example

You design StayHub chat; Lina checks

Design guest↔hotel chat for the 10× plan: 18,000 hotels with ~2 staff devices each, guests on the StayHub app, text and photos, delivery ticks, messages under a second when both are online, nothing lost when one is offline.

  1. 1Pin the requirements

    Functional: one conversation per stay (guest + all the hotel's devices), text and photos, ticks, history for both sides. Non-functional: delivery < 1 s when online; a message with one tick is never lost; offline devices catch up; history kept 2 years.

    Why: "Never lost after one tick" decides when we ack. "All the hotel's devices" decides that delivery is per device, not per person.

  2. 2Estimate messages and connections

    Messages: 25,000 bookings × 40% × 12 = 120,000/day ≈ 1.2/s, ~4/s in the evening. Connections: 36,000 staff devices + ~14,000 guests ≈ 50,000 open sockets. Storage ≈ 80 GB over 2 years with copies.

    text
    messages:    120,000/day ÷ 10⁵ s ≈ 1.2/s  → PostgreSQL is plenty
sockets:     36,000 + 14,000 ≈ 50,000     → 2 gateways at 25k, run 3
polling alt: 50,000 ÷ 10 s = 5,000 req/s  > 1,600 our app tier serves
storage:     ≈ 80 GB / 2 years            → no new database

    Why: The numbers say the message rate is trivial and connections are the real load, so the effort goes into the gateways, not the store.

  3. 3Give connections their own tier

    3 small gateway processes hold the WebSockets, behind the load balancer. They authenticate, pass frames to the chat module in the app, and push messages down. They're deployed separately from the app and drained slowly.

    Why: App deploys happen several times a day. If sockets lived on the app servers, every deploy would drop and reconnect all of them at once.

  4. 4Route between gateways

    Each gateway writes device → gateway to Redis with a 90 s TTL, refreshed by a 30 s heartbeat. To deliver, the chat module looks up the hotel's devices and publishes to each gateway's channel (Redis ).

    Why: Devices move between gateways on every reconnect. A TTL means a crashed gateway's entries disappear on their own within 90 s.

  5. 5Store first, then tick, then deliver

    Insert with seq and client_msg_id (unique per conversation), ack one tick, then deliver. Delivery is ; devices drop anything with a seq they already have.

    Why: Acking before storing turns a gateway crash into a lost message the guest believes was sent. The unique id makes resends harmless.

  6. 6Handle offline and reconnects

    No live socket → . On reconnect, after a random delay of 1–30 s, the device asks for messages WHERE seq > lastSeq. Photos go straight to object storage with a signed URL; messages carry only the reference.

    Why: Offline becomes a normal resync instead of a special case, and spreading reconnects avoids a herd of thousands at once.

The whole thing in one line: You found that 1.2 messages a second is trivial but 50,000 open sockets aren't, gave sockets their own gateway tier with a Redis registry, stored before every tick, made resends harmless with a client id, and turned offline into a jittered resync after the last sequence number.
ConcernChoice for StayHubWhy
TransportWebSocket, long polling as a fallbackTwo-way, low overhead; fallback for hotel networks that block it.
Message storePostgreSQL, key (conversation_id, seq)~1.2 writes/s; every read is "recent messages in one conversation".
Fan-outOn write, to each device of the stayA conversation has a guest and a few hotel devices: tiny, bounded.
PresenceCoarse: "front desk online" from heartbeatsExact per-person presence costs more than it's worth here.
MediaDirect upload to object storage, CDN downloadKeeps megabytes off the gateways.
Key choices for StayHub chat, each tied to an access pattern or number.
11

Interactive simulation

12

Visual diagram

Click a component

Ana and Bruno are connected to different gateways. Storage happens before delivery; push covers the offline case.

13

Analogy

  • An old phone switchboardEach person's open phone lineA chat systemA WebSocket to a server
  • An old phone switchboardThe directory of who's on which lineA chat systemThe connection registry
  • An old phone switchboardOperators passing messages alongA chat systemServers routing to each other
  • An old phone switchboardThe post office for absent peopleA chat systemDurable message storage
  • An old phone switchboardA "you have mail" postcardA chat systemA push notification
  • An old phone switchboardSigning for a tracked parcelA chat systemA delivery receipt

Where the analogy breaks: one person can have a phone, a laptop and a tablet all connected at once. Every message must reach each of them, in order.

14

Check your understanding

Q1/3Reasoning
Peak concurrent connections are 10M. Each gateway safely holds 250k connections. How many gateways do you need before adding headroom?
gateways
How confident are you?
15

Practice ladder

Step 1 · Trace

Trace a message through a tunnel

Scenario Warm-up

Using the StayHub chat design, trace what happens, step by step, and what each side sees:

  1. 18:00:00. A guest on a train sends "Arriving 21:00" (clientMsgId c-91). The chat service stores it as seq 14 in the stay's conversation, but the train enters a tunnel and the ack never reaches the phone.
  2. 18:00:20. Out of the tunnel, the app resends c-91.
  3. Meanwhile the hotel's tablet is online on gateway 2, and the receptionist's phone is offline; it reconnects at 18:30 with lastSeq = 12.
Step 2 · Complete

Finish the gateway's delivery code

Code Practice

Gateway 2 receives delivery requests from the chat service through Redis pub/sub. Lina's version has two gaps. Fill them in so that: a missing socket falls back to push, and a device never shows the same message twice.

// On the gateway: called for each delivery published to this gateway's channel
export async function deliver(deviceId: string, m: StoredMessage) {
  const socket = sockets.get(deviceId);
  if (!socket) {
    // ___ gap 1: the device isn't here (moved or offline) ___
    return;
  }
  socket.send(JSON.stringify({ type: "msg", ...m }));
}

// On the device (app code)
function onMessage(m: StoredMessage) {
  // ___ gap 2: skip anything we've already shown ___
  show(m);
  lastSeq[m.conversationId] = Math.max(lastSeq[m.conversationId] ?? 0, m.seq);
  ack(m.conversationId, m.seq);
}
Step 3 · Solve

Live booking alerts for hotels

Design Practice

The Hotel Partners team wants the front-desk tablet to show a new booking banner within 5 seconds of a guest booking (today the dashboard reloads every 2 minutes). At the 10× plan: 25,000 bookings/day, 18,000 hotels × ~2 devices.

Design it, reusing what you can from the chat design. Cover: transport, routing, what happens when a tablet is offline, and what happens on reconnect. Explain why you did or didn't need anything chat needed.

Step 4 · Decide

Read receipts for hotels?

Decision Stretch

Maya wants blue ticks: guests see when the hotel has read their message. The Hotel Partners team pushes back: "Hotels will feel watched, and a read message without a reply looks rude."

Facts: - A read receipt is one more small event per message per device: ~120,000 messages/day → a few extra events per second at most. - 22% of guest complaints last month were "the hotel never answered". - Front desks are shared: several staff read on one tablet.

Write your decision: "We chose X over Y because…, accepting the risk of…", including what "read" should mean for a shared front desk.

16

Explain it yourself

Walk through what happens when Ana sends a message to a group of 50 people, half of whom are offline.

0 words