Operating systems, processes and threads
Monday the 1st, 09:00: hotel owners start downloading their monthly reports, and guest searches jump from 150 ms to 4 seconds. The servers' CPU reads only 30%.
Your server program is never alone. Here's how the operating system keeps programs apart, shares a few CPU cores between many tasks, and where it gets risky.
- Problem
One machine with 4 to 64 CPU cores must run hundreds of programs and thousands of tasks at once, without them corrupting each other.
- Why the simple system fails
If every program could touch all memory and grab the CPU for as long as it liked, one bug or one busy loop would crash or freeze everything else.
- Engineering concept
Processes isolate memory; threads share it for cheap cooperation; a scheduler time-slices cores so everything appears to run at once.
- Technology
The kernel, virtual memory, system calls, the scheduler, process models (multi-process, thread pool, event loop).
- Trade-offs
Processes are safe but heavier; threads are cheap but need locks and invite race conditions; more threads than cores add switching overhead, not speed.
The problem
2 min- Omar · On-call SRESearch p99: 4 s. Normal is 150 ms. CPU on both servers: about 30%. Database fine.
- Karim · CTO30% CPU and it's slow? Last time slow meant full.
- Lina · Senior engineer, your mentorMake a guess before I say anything.
The big idea
4 minA core runs one thread at a time. Long computing on a shared thread makes everyone behind it wait.
The server isn't full; a few threads are. More servers won't help. Moving the computing somewhere else will.
How it works
5 minThe is the program that shares one machine between many programs. Here's how it runs StayHub:
- 1One machine, 8 cores
Each core runs one (a line of work) at any instant.
- 28 Node processes
We start one per core: a running program with its own private memory. If one crashes, the others keep serving.
- 3One main thread each
Each process runs our code on one thread with an : while a request waits on the database, it serves another.
- 4The scheduler shares cores
The gives each thread turns on a core. A thread that's waiting gives its turn away.
- 5Computing doesn't give way
A PDF render never waits, so it holds its thread for 1.5 s. Every request queued on that process waits too.
On the 1st, each report pins one process's thread for 1.5 s. Three of eight cores at 100%, five nearly idle: the graph shows 30-something percent. Searches sent to those three processes wait, sometimes behind two reports: that's the 4 s p99.
Watch a decision being made
4 minLina unfreezes search on the 1st
Every 1st, search p99 jumps from 150 ms to 4 s while CPU reads 30%. Find what's blocking and fix it, without buying servers.
- 1Look per process
pssorted by CPU: twonodeprocesses at ~100%, the rest at a few percent.bash$ ps -o pid,%cpu,rss,cmd -C node --sort=-%cpu PID %CPU RSS CMD 4121 99.6 412300 node server.js 4127 98.9 398120 node server.js 4124 3.1 210400 node server.jsWhy: A machine average hides single busy threads.
- 2Classify the work
A profile shows the time in
renderPdf. Fetching report rows is waiting (the event loop serves others meanwhile). Rendering is computing: 1.5 s with no wait inside.Why: Waiting is cheap on an event loop; computing blocks it.
- 3Give the computing its own process
The endpoint puts a job on a queue and answers "preparing your report". A separate worker process renders and stores the PDF; the owner gets a link.
Why: The worker gets its own core and memory. Guests never wait behind it, and a crash stays inside it.
- 4Measure on the next 1st
Search p99 back to about 150 ms; owners wait a few seconds for their report.
Why: Proves the block is gone and that the new cost is acceptable.
In real life
3 minGoogle Chrome: many processes
- Has publicly described running sites in separate processes
- A crashing page doesn't take down the whole browser
- Costs more memory, since processes don't share it
Worth it when one part failing must not hurt the rest.
Redis: one main thread for commands
- Its documentation describes running commands one at a time on one main thread
- No locks needed; one slow command blocks every client
Worth it when every task is tiny.
What goes wrong
3 minEach process has its own copy; they disagree and vanish on restart.
Data that must agree goes in a database or Redis.
A few cores can be pegged by single threads.
Look at CPU per process and per core.
Only as many threads as cores run at once; the rest take turns and pay for switching.
About one busy thread per core for computing; more only for work that mostly waits.
Trade-offs
3 min| Option | Fits when | You pay | Main risk |
|---|---|---|---|
| Several processes | Node, Python; heavy or crash-prone work | More memory, no sharing | Copies that disagree |
| Threads in one process | Java, Go, .NET; shared in-memory state | Locks, careful code | Races; one crash kills all |
| Event loop | Work that mostly waits (APIs) | One process per core | Any heavy computing blocks it |
- Karim · CTONode has worker threads. Render PDFs in a thread in the same process. Done today.
- Lina · Senior engineer, your mentorA thread shares the process. A 600-page report that runs out of memory takes down every guest on it.
- Karim · CTOYour separate worker means a queue, a new deploy and another thing to watch.
Your job as tech lead
4 minClaude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.
- Which work may run inside the web processes: quick request handling yes, a 1.5 s PDF render no.
- What may live in a process's memory. 16 copies mean 16 versions of the truth: prices and availability live in PostgreSQL or Redis.
- Which metric you trust for "is it busy": per-process CPU, not the machine average that showed 30% while search took 4 s.
On the 1st at 09:00 owners download monthly PDF reports. Each render takes about 1.5 s of CPU on the event loop of one of our 16 web processes, and search p99 jumps from 150 ms to 4 s. Move rendering out: the endpoint queues a job and returns right away; a separate worker process renders and stores the PDF. Don't change searches or bookings. Plan first, then a load test running 50 PDF requests alongside searches, with search p99 before and after.
- Ask where the PDF code runs now: "in the worker process", not "in the web process, but async".
- In the load test, search p99 stays close to 150 ms while PDFs are made.
- The diff adds no in-memory Map holding prices or rooms left.
- It wraps the render in
async/awaitand calls the event loop unblocked. - It raises threads or processes far beyond the 8 cores per server.
- It fixes an overbooking race with a lock inside one process, while 16 processes serve requests.
Check yourself
2 mintop shows the node process at 100%. What's going on?Next episode: Karim wants to double the app servers before summer. You lead this one: find which of CPU, memory, disk or network is actually slowing StayHub down.
Mark it complete to update your skill map.
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Check your understanding, Practice ladder, Explain it yourself.
Technical deep dive
Lina unfreezes search on the 1st
Every 1st of the month, search p99 jumps from 150 ms to 4 s while CPU reads 30%. Find what's blocking and fix it, without buying servers.
- 1Look per process
pssorted by CPU on one server: two or threenodeprocesses at ~100%, the rest at a few percent.bash$ ps -o pid,%cpu,rss,cmd -C node --sort=-%cpu PID %CPU RSS CMD 4121 99.6 412300 node server.js 4127 98.9 398120 node server.js 4124 3.1 210400 node server.jsWhy: A machine average hides single busy threads. Per-process numbers show them.
- 2Find what the busy thread is doing
Profile one pegged process for 10 s (a CPU profiler, or the slow-request log): almost all the time is in
renderPdf.Why: Knowing which code holds the thread tells you what to move.
- 3Classify the work
Fetching report rows is waiting (the event loop serves others meanwhile). Rendering is computing (1.5 s with no wait inside). Only the rendering is the problem.
Why: Waiting is cheap on an event loop; computing blocks it. The fix depends on which one you have.
- 4Give the computing its own process
The report endpoint puts a job on a queue and answers "preparing your report". A separate worker process renders PDFs and stores them; the owner gets a download link.
Why: The operating system gives the worker its own core and memory. Guest requests never wait behind it, and a crash stays inside the worker.
- 5Measure on the next 1st
Search p99 back to about 150 ms; the worker's core is busy, the app processes are not. Report wait for owners: a few seconds.
Why: Proves the block is gone and shows the new cost (owners wait a little) is acceptable.
Threads share their process's memory. That's fast, and it's exactly what makes them dangerous. Even single-threaded Node has the problem: every await is a point where another request can run in between.
// In-memory stock shared by every request in this process.
const roomsLeft = new Map<string, number>([["porto-9-2026-11-14", 1]]);
async function book(roomKey: string, guest: string) {
const left = roomsLeft.get(roomKey) ?? 0; // both requests read 1
if (left === 0) throw new Error("Sold out");
await chargeCard(guest); // await: the other request runs now
roomsLeft.set(roomKey, left - 1); // both write 0
return "confirmed"; // two confirmations, one room
}| Process | Thread | Event loop (one thread) | |
|---|---|---|---|
| Memory | Private | Shared with its process | Shared (one thread) |
| Creation cost | ~ms, MBs of memory | ~10–100 µs, ~1 MB stack | A callback: almost free |
| A crash affects | Only itself | The whole process | The whole process |
| Uses several cores | Yes, one per process | Yes | No: run several processes |
| Typical example | PostgreSQL connections, Chrome tabs | Java and Go servers | Node.js, Redis |
Interactive simulation
Visual diagram
One Linux server: isolated processes above, the kernel in the middle, shared hardware below.
Analogy
- A big restaurantThe head chef assigning stationsA serverThe operating system
- A big restaurantSeparate kitchens with their own fridgesA serverProcesses with private memory
- A big restaurantCooks sharing one fridgeA serverThreads sharing memory
- A big restaurantFour stoves, twenty cooksA serverFour cores, many threads
- A big restaurantTwo cooks grabbing the last eggA serverA race condition
Where the analogy breaks: a chef swaps cooks every few minutes. The OS swaps threads hundreds of times per second, so fast that you never see it, unless you create far too many.
Check your understanding
Practice ladder
Trace three requests on one process
Scenario Warm-upOne Node process (one main thread) receives three requests in this order, a few milliseconds apart:
- Search at t = 0: 5 ms of computing, then waits 60 ms on the database, then 5 ms of computing.
- Thumbnail resize at t = 2 ms: 400 ms of pure computing, no waiting.
- Hotel page at t = 4 ms: 3 ms of computing, then waits 20 ms on the database, then 2 ms of computing.
Trace what the thread does over time, and estimate when each request finishes.
Finish Lina's note on the image uploads
Debugging PracticeHotel owners upload room photos; StayHub resizes each one into 4 sizes inside the upload request. On Tuesday afternoons (when owners update listings), guest pages slow down. Lina's note:
- Look per process: 2 of 8 Node processes at ~100% CPU during uploads; machine CPU 27%.
- Find what the busy thread is doing: the profiler shows 90% of the time in
resizeImage. - Classify the work: ___
- Give the computing its own process: ___
- Measure next Tuesday: guest page p99 and how long owners wait for their photos.
Fill in steps 3 and 4.
The booking counter that drifts
Debugging PracticeMaya's new dashboard shows "bookings today". The code keeps a counter in each app process:
let bookingsToday = 0;
app.post("/bookings", async (req, res) => { /* …book… */ bookingsToday++; });
app.get("/stats/today", (req, res) => res.json({ bookingsToday }));StayHub runs 16 app processes on 2 servers. Maya says: "Every refresh shows a different number, all of them too low, and at 14:00 it went back to near zero." There was a deploy at 14:00.
Explain all three symptoms and propose a fix.
More processes or a worker?
Decision StretchOn the 1st, report rendering still competes with guests on the same machines. Two proposals:
- Omar: run 16 Node processes per 8-core server instead of 8. "More processes, more chances a search lands on a free one."
- Lina: keep 8 processes per server, and run PDF rendering in one separate worker process on its own small machine (about €40 a month).
Facts: each PDF takes 1.5 s of CPU; on the 1st, up to 4 reports render at the same moment; guest search needs 20 ms of CPU.
Write a short decision: "We chose X over Y because…, accepting the risk of…"
Explain it yourself
Explain the difference between a process and a thread, and why a CPU-heavy task can freeze a Node.js API but not a PostgreSQL server.