When not to use Kubernetes
Monday, 10:00: the three-person Experiences team has spent three weeks building its own Kubernetes cluster and shipped zero features. Launch is in five weeks, and Karim asks you to decide.
Kubernetes is powerful and expensive to run. Here's a simple way to decide whether you need it, or something much lighter.
- Problem
A 6-engineer hotel-tech startup is told by a new hire that "real companies run Kubernetes". The product is a web app, an API, a worker and a database.
- Why the simple system fails
Adopting Kubernetes by default turns a product team into a part-time platform team: cluster upgrades, ingress, certificates, autoscaling, security policies and YAML, before a single feature ships. Refusing it by default fails the opposite way when 40 teams each reinvent deployment.
- Engineering concept
Choose the least operationally expensive runtime that meets your actual constraints, and write down the triggers that would make you move up.
- Technology
PaaS, serverless functions, managed container services, managed Kubernetes, self-managed Kubernetes; operators/CRDs and service meshes as advanced Kubernetes add-ons.
- Trade-offs
Every step up the ladder buys control and flexibility with people-time. The right answer changes as team size, workload variety and scale change.
The problem
2 min- Karim · CTOExperiences: 3 engineers, 3 weeks spent on a Kubernetes cluster, 0 features shipped. Launch is in 5 weeks.
- Maya · Product manager2,000 beta users are waiting for booking. They don't care where it runs.
- Lina · Senior engineer, your mentorYou lead this one. I'll just check your answer.
The big idea
4 minChoose the lowest rung that meets your hard needs, count people's time as cost, and write down what would make you move up.
StayHub's cluster exists because 14 services and 25 engineers needed one platform, and two people run it. Experiences has 3 services and 3 people. Same tool, different constraints, different answer.
How it works
5 min- 1List hard needs
GPUs? On-premises? Compliance rules like encrypted service-to-service traffic? Long-running jobs? Private networking? Experiences: none.
- 2List constraints
3 engineers, nobody for platform work, launch in 5 weeks, a small budget.
- 3Find the lowest rung
Functions → → managed containers → managed Kubernetes → self-managed Kubernetes. A PaaS runs your code and patches everything else; managed containers add private networking without a cluster. Functions are too limited for a worker; a PaaS with a managed database meets every need.
- 4Count the full cost
Total cost of ownership is the cloud bill plus people's time for upgrades, on-call and incidents. A cluster can be cheaper per CPU and still cost far more.
- 5Record triggers
Write what would make you move up: more than 8 services, private networking to StayHub's database, a PaaS bill above €3,000 a month, or a workload the PaaS can't run.
Watch a decision being made
4 minYou write the Experiences runtime decision
Decide where Experiences (web, API, worker, Postgres; 2,000 beta users; 3 engineers; launch in 5 weeks) should run, and record it so it can be revisited on facts. Lina only reviews.
- 1Requirements and constraints
A Node web app, an API, a worker, PostgreSQL with daily backups. No GPUs, no on-premises, no special compliance. 3 engineers, none with platform experience; ~€1,000 a month budget.
Why: Hard requirements eliminate options; constraints decide how much platform work is affordable. Here: almost none.
- 2Compare three options
Own cluster, join StayHub's main cluster, or a PaaS with managed Postgres.
Why: The lowest workable rung plus one or two above it keeps the decision honest without a 10-option table.
- 3Cost both ways
Infra + people per month.
textOption Infra People Total/month Own cluster 700 4,000 ~4,700 Join main cluster 250 1,500 ~1,750 PaaS + managed DB 450 0 ~450Why: Lina: "the people column is the one everybody leaves out, and it's usually the biggest."
- 4Decide and record triggers
PaaS + managed Postgres. Revisit at more than 8 services, private networking to StayHub's database, a bill above €3,000 a month, or a workload the PaaS can't run.
Why: "Without triggers, this gets re-argued every month. With them, anyone can check."
In real life
3 minShopify: Kubernetes with platform teams
- Has publicly described running its platform on Kubernetes at very large scale
- Many services and teams, with dedicated platform engineers
- Huge sales peaks make consistent scaling worth it
Worth it with many teams and a platform group to run it.
37signals (Basecamp, HEY): no Kubernetes
- Has publicly described deploying with Kamal, a simple tool it built, onto its own servers
- A small team running a few large applications
- Chose fewer moving parts over a general platform
Worth it when a few big apps don't need a general platform.
What goes wrong
3 minStorage zones, backups, failover and upgrades become your problem.
A managed database, unless you have a strong reason and the expertise.
You pay the operating cost now for a scale you may never reach.
Start on the lowest rung that works; write down the triggers to move up.
One engineer's time can cost more than the entire PaaS bill.
Compare total cost: infrastructure plus people, on-call and upgrades.
Trade-offs
3 min| Option | Monthly (infra + people) | Gains | Costs |
|---|---|---|---|
| Own cluster | ~€4,700 | Full control and isolation | Every team rebuilds the platform |
| StayHub's main cluster | ~€1,750 | One platform, private network to other services | Load on Lina and Omar; shared blast radius |
| PaaS + managed DB | ~€450 | Fastest to production, nothing to run | Networking limits, a second way to deploy |
- Karim · CTOWe already have a cluster. Put Experiences in it: one platform, and if it grows, it's already there.
- Lina · Senior engineer, your mentorOmar and I run it a third of our time for 14 services. You'd add a fourth team's requests and on-call.
- Karim · CTOAnd if Experiences crashes the cluster in its first month, bookings go down with it.
Your job as tech lead
4 minClaude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.
- Which runtime rung Experiences gets: PaaS + managed Postgres (~€450/month, ~0 platform time) over its own cluster (~€4,700 with people time).
- What counts as a real trigger to move up: many services and teams, special workloads, or a bill bigger than the platform team you'd need. Not "big companies use it".
- Where the database lives: managed, not PostgreSQL in a cluster on a zonal volume.
Set up hosting for StayHub Experiences on our PaaS instead of the team's Kubernetes cluster: the web app, API and worker as three services from our existing Dockerfiles, plus a managed PostgreSQL, in an EU region. Use the platform's built-in TLS, health checks, rollbacks and autoscaling; don't add Helm or anything we'd have to operate. Secrets come from platform settings, never repo files. Plan first: what the platform does vs what we still own, the monthly cost, and the limits that would make us outgrow it. Then write a short ADR with those triggers.
- The list "what we still own" is short.
- The monthly estimate is near ~€450, and Claude says where each number comes from.
- The ADR names concrete triggers, not "when we scale"; deploy once and roll back once to see both work.
- It recommends Kubernetes "to be ready for scale" with no requirement the PaaS can't meet.
- It compares only infrastructure bills and leaves out engineer time.
- It runs the production database in a cluster "so everything is in one place".
Check yourself
2 minNext module: StayHub is heading for a million guests a day, and search feels slow even at 3 a.m., when nobody else is using it.
Mark it complete to update your skill map.
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
Technical deep dive
You write the decision record. Lina only reviews the final version; her two comments are in the "why" lines.
You write the Experiences runtime decision
Decide where the Experiences product (web, API, worker, Postgres; 2,000 beta users; 3 engineers; launch in 5 weeks) should run, and record it so it can be revisited on facts.
- 1List the hard requirements
Must run a Node web app, an API, a background worker, and PostgreSQL with daily backups. No GPUs, no on-premises, no compliance rule beyond StayHub's usual GDPR basics, no long-running jobs.
Why: Hard requirements eliminate options. Everything else is preference, and preferences shouldn't cost 3 weeks.
- 2List the constraints
3 engineers, none with platform experience; launch in 5 weeks; budget around €1,000 a month for infrastructure.
Why: Constraints decide how much platform work is affordable. Here: almost none.
- 3Compare three options
Own cluster, join StayHub's main cluster, or a PaaS with managed Postgres. Drop functions (too limited for a worker) and self-managed Kubernetes (no team).
Why: Comparing the lowest workable rung with one or two above it keeps the decision honest without a 10-option table.
- 4Cost both ways
Infra + people per month: own cluster ~€700 + ~€4,000; main cluster ~€250 + ~€1,500 of Lina and Omar's time; PaaS ~€450 + ~€0.
textOption Infra People Total/month Own cluster 700 4,000 ~4,700 Join main cluster 250 1,500 ~1,750 PaaS + managed DB 450 0 ~450Why: Lina: "the people column is the one everybody leaves out, and it's usually the biggest."
- 5Decide and record triggers
Decision: PaaS + managed Postgres. Revisit if: more than 8 services, private networking to StayHub's database is needed, the PaaS bill passes €3,000 a month, or a workload the PaaS can't run appears.
Why: Lina: "good. Without triggers, this gets re-argued every month. With them, anyone can check whether it's time."
| Option | You operate | Typical team fit | Strong reasons to choose | Main limits |
|---|---|---|---|---|
| PaaS | Code, config | 1–30 engineers, few services | Speed, zero ops, built-in rollbacks | Networking, special hardware, unit cost at scale |
| Serverless functions | Functions, triggers | Any size, event-shaped work | Spiky load, scale to zero, pay per use | Duration limits, cold starts, DB connection storms |
| Managed containers | Images, service config, networking | 5–80 engineers | Any runtime, private networking, no cluster | Less ecosystem; some advanced patterns missing |
| Managed Kubernetes | Everything above the control plane | Many services and teams, platform team of 2+ | Consistency across many services, special workloads, portability | People cost, upgrades, security surface, complexity |
| Self-managed Kubernetes | Everything | Large infra teams, on-premises | Regulation, sovereignty, own hardware | Highest operational risk |
Interactive simulation
Visual diagram
The runtime ladder: each step right adds control and operational work.
Analogy
- Feeding hotel guestsA catering contractHosting your codePaaS
- Feeding hotel guestsOrdering each dish à la carteHosting your codeServerless functions
- Feeding hotel guestsRenting an equipped kitchen by the hourHosting your codeManaged containers
- Feeding hotel guestsBuilding your own kitchen in a rented buildingHosting your codeManaged Kubernetes
- Feeding hotel guestsOwning the building tooHosting your codeSelf-run Kubernetes
A 20-room hotel builds a kitchen only when catering can't do what it needs, not because the Ritz has one. Where the analogy breaks: you can switch caterers in a week, but moving software between platforms can take months.
Check your understanding
Select all that apply.
Practice ladder
Trace the full cost
Estimation Warm-upThe Experiences team kept a log of their first 3 weeks on their own cluster. Each engineer costs about €10,000 a month.
| Work | Hours |
|---|---|
| Cluster + node pools | 30 |
| Ingress + certificates + DNS | 40 |
| Monitoring + alerts | 35 |
| Helm charts for 3 services | 45 |
| Debugging networking | 30 |
Infrastructure bill for the month: €700.
- How much did the 3 weeks of platform work cost in people's time (assume ~160 working hours per engineer per month)?
- Compare with a PaaS at €450 a month that needs about 10 hours of setup.
Finish the decision record
Decision PracticeHere's your decision record for Experiences. Two sections are empty. Fill them in.
Context: new product, 3 engineers, web + API + worker + Postgres, 2,000 beta users, launch in 5 weeks. Hard requirements: daily database backups; no GPUs; no on-premises; GDPR basics. Options considered: own cluster (~€4,700/month all-in), main cluster (~€1,750), PaaS + managed Postgres (~€450). Decision: ___ Triggers to revisit: ___
Where should the pricing model run?
Decision PracticeStayHub's data team built a model that suggests room prices. It needs a GPU for about 2 hours every night, and must read from StayHub's main database through private networking. The data team is 2 people. StayHub's main cluster has no GPU nodes yet.
Options: a PaaS (no GPUs offered), serverless functions (15-minute limit), a GPU node pool in the main cluster that scales to zero during the day, or a dedicated GPU VM running 24/7 (~€1,800 a month).
Pick one and justify it with hard requirements, constraints and cost.
A service mesh for the main cluster?
Decision StretchA new engineer proposes adding a service mesh to StayHub's main cluster: "mTLS between all services, retries, and great observability for free."
Facts: - 14 services, about 120 pods. A sidecar proxy uses about 60 MiB of memory and a little CPU per pod, and adds about 1 ms per hop. - No compliance rule requires encrypted service-to-service traffic today; card data is handled only by payments-api, which talks to the provider over TLS. - Lina and Omar run the cluster with about a third of their time; upgrades already take 2 days a quarter. - Tracing is already in place for the main paths.
Write a short decision: "We chose X over Y because…, accepting the risk of…".
Explain it yourself
Your CTO asks: "Should we move to Kubernetes?" Explain how you'd answer, without saying yes or no up front.