The scale journey: 100 to 100 million users
Thursday, 14:00: StayHub's investors want a plan for ten times the guests and two new continents within a year. Karim's draft slide says "microservices, Kafka, multi-region". You present on Monday.
Big systems aren't designed in one go. They grow one piece at a time, each time something starts to break.
- Problem
Your app is growing 10× a year. You can't rebuild it at every step, and you can't afford to build for 100M users on day one.
- Why the simple system fails
Copying a big company's architecture early burns money and team time on problems you don't have; never evolving means outages exactly when growth arrives.
- Engineering concept
Scale in steps. At each traffic level, measure, find the one component that saturates first, and apply the cheapest change that removes it.
- Technology
Separate DB, CDN, load balancer + stateless servers, cache, queues and workers, read replicas, search engine, sharding, multi-region.
- Trade-offs
Every step adds cost, moving parts and new failure modes (stale cache, replica lag, cross-shard queries). Adding one too early is waste; too late is an outage.
The problem
2 min- Karim · CTOInvestors want 10× guests (about 12 million a day) and launches in North America and Asia, within 12 months.
- Karim · CTOMy draft slide: "microservices, Kafka, multi-region active-active". That's what the big ones do, right?
- Lina · Senior engineer, your mentorYou lead this one, start to finish. You present on Monday.
The big idea
4 minGrow one measured bottleneck at a time. A plan is the order in which walls arrive, with a trigger for each step.
Every step in this course fixed a measured problem, in the order it appeared. The next year's plan should look exactly like that, just written in advance.
How it works
5 min- 1Estimate the load
A back-of-the-envelope estimate in round numbers: 12M guests × 40 requests ÷ 86,400 s × 3 (peak) ≈ 16,700 req/s; the CDN serves ~70%, so ~5,000 reach us. Writes: ~6,000/s at peak.
- 2Compare with measured limits
Pods: add more. Reads: replicas and cache scale by copies. The leader: ~3,000 writes/s, measured in a load test.
- 3Order the walls
Leader writes hit their wall at ~5× today, mid-year. Asia's latency hurts now: 230 ms per round trip to Frankfurt.
- 4Pick the cheapest step for each
Bottleneck-driven scaling: change only what runs out first. Asia now: pods + a read replica in Tokyo, writes still in Frankfurt. Writes at ~4×: shard bookings by hotel_id.
- 5Attach triggers
A trigger is a measured threshold for the next step: writes > 1,500/s for a week → start sharding; Asia > 30% of bookings → plan Asia writes.
- 6Show the cost
Infra roughly ×4 over the year (not ×10, thanks to CDN, cache and replicas), plus 3 people on sharding for a quarter.
Watch a decision being made
4 minYou plan StayHub's next 10×
Turn "10× guests and two new continents in a year" into a sequenced plan with costs and triggers the team can execute. Lina checks twice.
- 1Estimate in round numbers
Average, peak, what reaches origin, and writes.
pythonavg = 12_000_000 * 40 / 86_400 # ~5,600 peak = avg * 3 # ~16,700 origin = peak * 0.3 # ~5,000 writes = 600 * 10 # ~6,000Why: Orders of magnitude are enough, and they keep the discussion about our system, not famous ones.
- 2Find the number above its limit
Pods: ~30 more at 175 req/s each. Reads: more copies. Leader: ~3,000 writes/s → over the limit at ~5×.
Why: Lina: "the only number that changes the architecture is the one above its limit. Everything else is more of the same."
- 3Order the walls, cheapest step each
- Asia latency (now): Tokyo read region. 2. Read load (~3×): more replicas. 3. Leader writes (~4–5×): shard by hotel_id. 4. Asia writes, only past 30% of bookings.
Why: Each choice reuses what the team already runs. Active-active writes stay out until something justifies their conflicts.
- 4Triggers with owners
Every trigger on a dashboard with an owner; sharding starts at 1,500 writes/s.
Why: Lina: "sharding takes months, so you start before the wall, not at it."
In real life
3 minShopify: a modular monolith
- Has publicly described keeping its core as one large, carefully modularised codebase
- Scaled data by splitting it by shop rather than splitting the code
- Fewer network hops and deploys to coordinate
Fits when most teams work on one product and data splits by tenant.
Amazon: many independent services
- Has publicly described moving to independent services owned by small teams
- Each team deploys and scales its own service
- Needs strong platform, monitoring and API discipline
Fits very many teams that must move independently.
What goes wrong
3 minYou pay for scale you don't have and inherit failure modes without their reasons.
Estimate your load, find your first wall, take the cheapest step.
Peaks of 3× or more hit capacity sized for the average.
Plan for measured peaks, with headroom.
Sharding or a new region takes months.
Set triggers below the limit, using each step's lead time.
Trade-offs
3 min| Asia launch | Gains | Costs |
|---|---|---|
| Full Tokyo region with writes | Every click fast, checkout too | A write home per hotel or double bookings; ~2× infra in Asia |
| Tokyo read region, writes in Frankfurt | Most latency gone, no conflicts | Checkout pays ~230 ms once |
- Karim · CTOAsia is the growth story. Launch a full Tokyo region now, writes included. Investors will love it.
- Lina · Senior engineer, your mentorReads in Tokyo fix most of the latency. Writes there mean each hotel needs a home region, or we get Experiences' double bookings.
- Karim · CTOCheckout from Tokyo still crosses to Frankfurt: 230 ms on the most important click.
Your job as tech lead
4 minClaude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.
- Which wall comes first, from your own numbers: ~6,000 writes/s at 10× against a leader's ~2,500–3,000.
- What must never be written in two regions at once: room inventory. An agent offering active-active writes is choosing double bookings for you.
- The order and timing of big projects: sharding and a second region start ahead of the wall.
I need a capacity estimate for StayHub at 10x before any architecture changes. Don't modify code or infrastructure; write a short report in docs/capacity-10x.md. Inputs: 12M guests/day, about 40 requests each, peak 3x the average, about 70% served by the CDN, peak writes today about 600/s; our load test showed one Postgres leader handles 2,500 to 3,000 booking writes/s. For each tier, show the maths, today's load, the load at 10x, and at what multiple it hits its limit. Sort by which breaks first. Mark every number as measured or assumed. No tool recommendations.
- Redo the headline maths: ≈ 5,600 req/s average, ≈ 16,700 at peak, ≈ 5,000 at origin.
- Every number is labelled measured or assumed.
- Primary writes come first in the order; no tool names you didn't ask for.
- It proposes microservices to "scale" without naming which tier is saturated.
- It designs active-active booking writes without mentioning the ~250 ms window between regions.
- It quotes a "Postgres max writes per second" from memory instead of our load test.
Check yourself
2 minNext module: the uptime dashboard says StayHub was 100% up last month, yet 300 guests couldn't pay on Saturday, and a hotel chain wants 99.99% in its contract.
Mark it complete to update your skill map.
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
Technical deep dive
You build the plan for Monday. Lina's two checks are in the "why" lines.
You plan StayHub's next 10×
Turn "10× guests and two new continents in a year" into a sequenced plan with costs and triggers, that the team can actually execute.
- 1Estimate the load
12M guests × 40 requests ÷ 86,400 s ≈ 5,600 req/s average; × 3 at peak ≈ 16,700; 30% reaches origin ≈ 5,000 req/s. Writes scale with bookings: ~600 → ~6,000/s at peak.
pythonavg = 12_000_000 * 40 / 86_400 # ~5,600 peak = avg * 3 # ~16,700 origin = peak * 0.3 # ~5,000 writes = 600 * 10 # ~6,000Why: Round numbers are enough to see orders of magnitude, and they keep the discussion about our system, not about famous ones.
- 2Compare with measured limits
Pods: 175 req/s each at 70% → ~30 more pods. Reads: cache + replicas scale by adding copies. Leader: ~3,000 writes/s measured → over the limit at ~5× today.
Why: Lina: "the only number that changes the architecture is the one above its limit. Everything else is more of the same."
- 3Order the walls
- Asia latency (now). 2. Read load (~3×). 3. Leader writes (~4–5×). 4. Asia writes (if Asia passes 30% of bookings). North America: CDN + read region like Asia, later in the year.
Why: Steps are scheduled by when each wall arrives, not by how impressive they sound.
- 4Choose the cheapest step for each
Asia: app pods + read replica in Tokyo, bookings still written in Frankfurt (one ~230 ms trip at checkout only). Writes: shard bookings by hotel_id with consistent hashing (planned last episode).
Why: Each choice reuses what the team already runs. Active-active writes are deliberately not in the plan until there's a reason that justifies their conflicts.
- 5Attach triggers and owners
Leader writes > 1,500/s for a week → start the sharding project (3 months lead time). Asia > 30% of bookings → design Asia writes. Each trigger on a dashboard, with an owner.
Why: Lina: "a trigger at 1,500, not 3,000: sharding takes months, so you start before the wall, not at it."
- 6Show the cost
Infra roughly ×4 over the year (not ×10, thanks to CDN, cache and replicas), plus a 3-person team on sharding for a quarter. Karim's slide would cost far more, sooner.
Why: Investors and Karim need money and people, not architecture words.
| Step | Measured signal that justifies it | New cost or failure mode |
|---|---|---|
| CDN | Static bytes dominate; distant users slow | Cache invalidation on deploys |
| LB + stateless servers | App CPU > 70% at peak; downtime on deploy | Sessions, holds, jobs must leave the pods |
| Cache | Same reads repeated; DB CPU rising with reads | Stale data, invalidation bugs, stampedes |
| Queue + workers | Slow side work inflates user latency | Eventual completion, retries |
| Read replicas | Primary saturated by reads despite cache | Replication lag, read-your-writes bugs |
| Sharding | Writes or data size beyond one leader | Cross-shard queries, resharding, more databases |
| Multi-region | Distant users' latency; regional outage unacceptable | Cross-region consistency, roughly 2× infrastructure |
Interactive simulation
Visual diagram
The architecture at roughly 1M users. The walkthrough adds each component in the order it usually becomes necessary.
Analogy
- From food truck to restaurant chainOne cook does everything in a food truckFrom one server to a big systemOne server runs code and database
- From food truck to restaurant chainThe pantry gets its own locked roomFrom one server to a big systemThe database moves to its own machine
- From food truck to restaurant chainSeveral cooks, a host seats guestsFrom one server to a big systemLoad balancer and identical servers
- From food truck to restaurant chainPopular sauces made in advanceFrom one server to a big systemA cache for common answers
- From food truck to restaurant chainBranches open in new citiesFrom one server to a big systemRunning in several regions
Where the analogy breaks: a restaurant can't copy its pantry instantly. Software can copy data in milliseconds, but those copies can disagree for a moment, which is a whole new kind of problem.
Check your understanding
- 1.Stateless app servers behind a load balancer
- 2.Go multi-region
- 3.Move the database to its own managed instance with backups
- 4.Shard the largest tables
- 5.Cache hot reads and move slow work to a queue
- 6.Add read replicas
Practice ladder
Trace the plan's numbers
Estimation Warm-upFor the North America launch, Maya expects 3 million guests a day there within a year, 35 requests per guest per day, peak at 4× the daily average (evenings are concentrated), CDN serving 70%. One booking-api pod handles 175 req/s at the 70% target.
- Peak requests per second, and at origin.
- How many pods for North America alone?
- If 1 in 10 origin requests is a read that misses the cache and hits the database, how many database reads per second?
Match signals to steps
Decision PracticeLina's monitoring page lists signals. For each, name the step it justifies (or "none yet") and the risk that step brings.
- Primary CPU 80%, 94% of queries are SELECTs, cache hit ratio already 95%.
- Leader at 1,600 booking writes/s for a week; measured limit ~3,000.
- Asia: 18% of bookings; hotel-page p95 for Tokyo guests 1.4 s vs 0.3 s in Paris.
- App pods at 40% CPU at peak; database fine.
- Checkout p99 at 2.5 s because the invoice PDF is generated during the request.
Write the plan for a smaller company
Design PracticeA friend's startup, "CampFinder", books campsites. Today: 20,000 guests a day, one app server (CPU 55% at peak), one PostgreSQL (CPU 30%), no CDN, photos served by the app server (65% of bandwidth). Growth: 5× in a year. Summer weekends are 4× a normal day.
Write a one-year plan: ordered steps, the signal that triggers each, and what you would explicitly not do yet.
Kafka for the next year?
Decision StretchAn engineer proposes adding Kafka to StayHub this quarter: "Every service will publish events; it's what big companies do, and we'll need it at 10×."
Facts: - Today, 3 services need to react to new bookings (emails, owner notifications, analytics). They use the existing job queue, which handles ~600 jobs/s at peak with no backlog. - At 10×, that's ~6,000 jobs/s. The managed queue's documented limit is far higher. - Nobody on the team has run Kafka in production. - The sharding project starts next quarter with 3 engineers.
Write a short decision: "We chose X over Y because…, accepting the risk of…", with a trigger to revisit.
Explain it yourself
A founder asks: “What architecture do we need for 100 million users?” Explain how you'd answer, and walk through how the architecture would actually evolve.