Skip to content
Episode 34 · Sunday evening photo storm

Serverless: functions, cold starts and pay-per-use

Sunday, 19:20: hotels upload about 4,000 photos in two hours to prepare the week, the web servers spend their CPU resizing them, and search p95 jumps from 0.4 to 2.6 seconds.

Hand the cloud your code and let it run only when something happens. Great for quiet or spiky work, pricey for busy, steady work.

30 min intermediateServerless functions (FaaS)Cold startsPay-per-use vs provisioned costAutoscalingManaged services vs self-hostingStateless servers
  1. Problem

    Traffic is spiky or tiny: a webhook fires 200 times a day, an image resize runs after uploads, a marketing page gets 50x traffic during a campaign.

  2. Why the simple system fails

    Always-on servers are paid 24/7 even when idle, must be sized for the peak, and need patching, scaling and monitoring by your team.

  3. Engineering concept

    Run code only when an event arrives. The platform creates instances on demand, scales them out per request and back to zero, and bills per invocation and execution time.

  4. Technology

    FaaS (Function as a Service) platforms, event triggers (HTTP, queues, object storage, schedules), managed databases and queues around them.

  5. Trade-offs

    No servers to run and near-zero idle cost, versus cold starts, execution limits, cost at steady high volume, harder local testing and vendor lock-in.

01

The problem

2 min
Sunday, 19:20. The weekly slowdown, again.
  1. Omar · On-call SRE
    Search p95 is 2.6 s (normally 0.4). Web servers at 95% CPU. About 4,000 photos uploaded since 19:00.
  2. Karim · CTO
    Two more servers just for Sunday? Or is this a job for serverless?
  3. Lina · Senior engineer, your mentor
    You lead this one. Commit to a guess.
02

The big idea

4 min
Pay for capacity, or pay for use?
The big idea

means paying per use instead of for capacity: perfect for bursty work nobody waits on, costly and slow for steady traffic guests wait on.

Always-on servers are a company car fleet: you pay for every car, busy or not. Functions are taxis: ten appear when ten people need them, and you pay per ride. A team commuting the same route all day is cheaper with its own cars.

03

How it works

5 min

With , you deploy code plus a trigger (an HTTP request, a file landing, a queue message, a schedule); the provider runs, scales and patches the servers. Photos arrive at about 13 an hour all week, then 2,000 an hour on Sunday evening: servers sized for Sunday are idle 98% of the time.

One photo, serverless
  1. 1
    Ask for an upload URL

    The dashboard gets a short-lived signed URL from the monolith, so the big file never passes through our servers.

  2. 2
    Upload straight to storage

    The browser sends the photo directly to , which emits an object-created event.

  3. 3
    One function per photo

    The platform runs resizePhoto per event: 4,000 photos can mean hundreds of instances at once.

  4. 4
    Save results, keep nothing

    Thumbnails go to storage, metadata to the database. The instance keeps no state; it may vanish any time.

  5. 5
    Back to zero

    After the burst, instances shut down: , no events, no cost. The next photo pays a : the delay while the platform starts a fresh instance.

04

Watch a decision being made

4 min
Worked example

You move resizing to a function (Lina checks the numbers)

Stop Sunday uploads from slowing search, without paying for idle capacity all week and without endangering the shared database.

  1. 1Measure the shape, do the cost math

    ~24,000 photos a month, ~1.5 s and 1 GB each, nobody waiting. 24,000 × 1.5 × 1 = 36,000 GB-seconds ≈ $0.60 a month at an illustrative price. Two extra Sunday servers: about $120.

    Why: "Serverless is cheaper" is only true for some shapes. Ten minutes of arithmetic turns a debate into a number.

  2. 2Design the trigger path, keep the handler thin

    Signed URL → direct upload to storage → object-created triggers resizePhoto. The resize logic stays in the plain photos module; the handler only reads the event and saves the result.

    Why: Big bytes never touch our servers, and provider lock-in stays in the thin glue.

  3. 3Protect the shared database

    One connection per instance, created outside the handler, through the pooler; a maximum concurrency of 50 for this function.

    Why: Functions scale out in seconds; PostgreSQL's 200 connections don't, and they're shared with every booking. With a cap, a burst queues a little instead.

  4. 4Make retries safe

    Output files are named by the photo's key and the update is SET ready = true, so running an event twice gives the same result.

    Why: Event triggers deliver at least once. Idempotent work turns duplicates into a non-event.

The whole thing in one line: Bursty and asynchronous: about $0.60 a month instead of $120, direct uploads, a thin handler, a concurrency cap and pooled connections, and harmless reruns.
05

In real life

3 min
Pay per use, or own the capacity

iRobot: a serverless cloud backend

  • Has publicly described running its robot cloud largely on serverless services
  • Traffic is very uneven through the day; a small team avoided running servers

Fits uneven, event-shaped traffic and a small ops team.

37signals: leaving the cloud for its own servers

  • Publicly described moving HEY and Basecamp to servers it owns
  • Large, steady traffic; reported big savings, taking on hardware work

Fits steady, predictable load where idle capacity is rare.

06

What goes wrong

3 min
Low-traffic checkout endpoints on functions without a plan

Instances go cold, so real guests hit seconds of p99.

Instead

Measure cold starts, or keep the endpoint always-on.

Assuming serverless is always cheaper

At steady high volume, per-call billing can cost several times more.

Instead

Model cost at expected and 10× volume first.

Long-running jobs in a single function

Execution limits kill it halfway, after side effects.

Instead

Idempotent steps through a queue, or a container job.

07

Trade-offs

3 min
OptionFits whenYou pay
FunctionsSpiky, low or event-driven work; tiny ops teamCold starts, time limits, cost at steady load, lock-in
Always-on serversSteady traffic, strict p99, long connectionsIdle capacity; you patch and scale
HybridA steady core plus bursty side jobsTwo deployment models
Monday, 11:00. Karim has seen the photo bill.
  1. Karim · CTO
    Sixty cents a month. Let's move the whole booking API to functions and stop paying for four servers.
  2. Lina · Senior engineer, your mentor
    It serves about 40 requests a second all day. Steady load costs more per call, and cold starts would land on checkout.
  3. Karim · CTO
    No servers to patch, no capacity planning. Engineer time is our most expensive resource.
08

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own
  • Which workloads go to functions, by traffic shape: Sunday photo bursts yes; the steady booking API with 0.3–1.5 s cold starts, no.
  • The cost comparison with your numbers (~$0.60 vs ~$120 a month), as arithmetic, not a claim.
  • The concurrency and connection budget: 200 connections, ~100 used, so the function gets a cap of 50 and the pooler.
Tell Claude Code
On Sunday evenings hotels upload ~4,000 photos and resizing on our web servers pushes search p95 from 400 ms to 2.6 s. Plan a move to a serverless function; no code until I approve. Signed upload URL from the monolith, direct upload to storage, object-created triggers resizePhoto. Resize logic stays in the photos module; thin handler. One pooled connection per instance, created outside the handler; max concurrency 50. Reruns must be idempotent. Show the monthly cost formula. Don't touch the booking API.
Check before you accept
  • The cost math: invocations × seconds × GB = GB-seconds, × price. 24,000 × 1.5 × 1 = 36,000.
  • A concurrency limit in the infrastructure file, and the connection created outside the handler through the pooler.
  • A test runs the handler twice on the same event: same files, same row.
Red flags
  • A new database connection inside the handler on every invocation.
  • Moving the steady booking API to functions "for scaling" without cold-start or cost numbers.
  • Business logic written inside the provider-specific handler.
09

Check yourself

2 min
Q1/3Debugging
A checkout function has p50 latency of 80 ms but p99 of 2.4 s. Traffic is ~3 requests per minute. What is the most likely cause?
How confident are you?

Next episode: a new engineer asks why StayHub uses an outbox, ports for payments and a photo function, and the only answers are in old chat threads.

Finished?

Mark it complete to update your skill map.

Next: Architecture Decision Records (ADRs) in practice
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Visual diagram, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.
10

Technical deep dive

Inside a function's life

You lead; Lina only checks the numbers. Here's how you'd move resizing off the web servers.

Worked example

Moving photo resizing to a function

Stop Sunday photo uploads from slowing search, without paying for idle capacity all week and without endangering the shared database.

  1. 1Measure the workload's shape

    ~4,000 photos Sunday 19:00–21:00, ~300 a day otherwise: about 24,000 a month. Each resize: ~1.5 s, 1 GB. Nobody waits for the result in real time.

    Why: Serverless wins on bursty, asynchronous work and loses on steady, latency-critical work. The shape decides before anything else.

  2. 2Do the cost math

    24,000 × 1.5 s × 1 GB = 36,000 GB-seconds. At an illustrative $0.0000167 per GB-second, about $0.60 a month, plus about half a cent in request fees. Two extra Sunday servers: about $120 a month.

    Why: "Serverless is cheaper" is only true for some shapes. Ten minutes of arithmetic turns a debate into a number.

  3. 3Design the trigger path

    Browser asks the monolith for a signed upload URL → uploads directly to object storage → each object-created event triggers resizePhoto.

    Why: The large bytes never touch our servers or the function's request path, which keeps both cheap and fast.

  4. 4Keep the handler a thin adapter

    Resize logic lives in the plain photos module; the handler only reads the event, calls the module and saves the result.

    Why: Same idea as last episode's ports: the business logic stays testable and portable, and lock-in stays in the thin glue.

  5. 5Protect the shared database

    One connection per instance, created outside the handler, through a pooler; reserve a maximum concurrency of 50 for this function so a burst queues instead of opening 400 connections.

    Why: Functions scale out in seconds; PostgreSQL's 200 connections don't. The database is shared with every guest's booking.

  6. 6Make retries safe

    The platform may run an event twice. Output files are named by the photo's key and the update is SET ready = true, so running twice gives the same result.

    Why: Event triggers deliver at least once. Idempotent work turns duplicates into a non-event.

The whole thing in one line: You checked the workload was bursty and asynchronous, showed it costs about $0.60 a month instead of $120, uploaded directly to storage with one function per photo, kept the handler thin, capped concurrency and connections, and made reruns harmless.
WorkloadFunction math (512 MB, 100 ms)Function cost / monthAlways-on alternative
Webhooks: 10k calls/day300k calls × 0.05 GB-s = 15k GB-s≈ $0.30≈ $30 for one small container, idle 99.9% of the time
API: 50 req/s steady130M calls × 0.05 GB-s = 6.5M GB-s≈ $135≈ $60–90 for 2–3 containers with headroom
API: 500 req/s steady1.3B calls → 65M GB-s≈ $1,350≈ $300–600 for a container fleet
Illustrative numbers only. The shape is general: pay-per-use wins at low or spiky load and loses at high steady load.
ConcernTypical behaviourMitigation
Cold startTens of ms to several secondsSmaller bundles, init outside the handler, provisioned concurrency for critical paths
Max execution timeMinutes, not hours, per callSplit work into steps, or use a container job
StateInstances are short-lived; memory may vanish anytimeKeep state in a database, cache, storage or queue
DB connectionsEach instance may open its ownPooler, 1 connection per instance, concurrency cap, queue
11

Visual diagram

Click a component

A serverless photo pipeline for a hotel platform. Every compute box runs only when an event arrives.

12

Analogy

  • Getting around townOwning a company car fleetRunning codeAlways-on servers
  • Getting around townCalling a taxi, paying per rideRunning codeServerless functions
  • Getting around townTen taxis for ten peopleRunning codeOne copy per request
  • Getting around townWaiting for a taxi to arriveRunning codeA cold start
  • Getting around townLoyalty points that don't transferRunning codeVendor lock-in

Where the analogy breaks: a taxi never forgets you mid-ride. A function copy can vanish between calls, so it must not keep anything important in memory.

13

Why it matters

Lots of work is quiet most of the time. A payment gets a few hundred calls a day. Photo thumbnails come in bursts on Sunday evenings. A nightly report runs for three minutes.

With your own servers, you either pay for the peak all month, or you fall over at the peak. And someone has to patch the machines and tune the scaling.

With functions, scaling and patching are the provider's job, and idle time is free.

Which traffic shape fits?

Spiky or quiet

  • Webhooks, thumbnails, nightly jobs
  • Idle time costs nothing
  • Bursts handled for you

Serverless shines

Busy all day

  • Constant high traffic
  • Per-call price adds up
  • Cold starts on urgent paths

Always-on servers are cheaper

14

Check your understanding

Q1/3Recall
"Serverless" means your code runs without any servers.
How confident are you?
15

Practice ladder

Step 1 · Trace

Trace warm and cold instances

Scenario Warm-up

StayHub's createUploadUrl function (called when a hotel starts an upload) has: - Cold start: 800 ms; handler: 30 ms - An idle instance is shut down after 10 minutes without requests - One instance handles one request at a time

Requests arrive at: 09:00:00, 09:00:05, 09:00:05 (two at once), 09:14:00.

For each request, say whether it hits a warm or cold instance, and its latency. How many instances exist at 09:00:06?

Step 2 · Complete

Finish the cost comparison

Estimation Practice

Lina's cost note for moving the payment-provider webhook handler to a function has two blanks. Use illustrative prices: $0.20 per million invocations and $0.0000167 per GB-second.

  • Webhooks: 6,000 per day (≈ 180,000 per month), 200 ms each, 512 MB.
  • Today they run on one always-on small server that does nothing else: $30/month.
  1. GB-seconds per month: ___
  2. Function cost per month (compute + requests): ___
  3. Decision and one risk to check.
Step 3 · Solve

Nightly partner reports

Design Practice

Every night at 02:00, StayHub must generate a PDF revenue report for each of the 1,800 partner hotels and email it. Each report takes ~4 s and ~300 MB. Today one worker process does them one by one: it takes 2 hours and sometimes isn't done when hotels check their email at 06:00 after a slow night.

Design a serverless version: the trigger, how work is split, what limits you set, how retries stay safe, and the rough monthly compute cost.

Step 4 · Decide

Search autocomplete on functions?

Decision Stretch

The Search team wants to put city autocomplete ("Par…" → Paris, Parma) on functions, to scale for a TV campaign next month.

Facts: autocomplete gets ~120 requests/s during the day, ~10/s at night; guests expect results in under 100 ms while typing; the campaign may bring 5× traffic for one evening. The handler is 15 ms; cold starts would be 300–900 ms. The monolith's 4 servers currently run at 35% CPU.

Write a decision: "We chose X over Y because…, accepting the risk of…", and say how you'd handle the campaign.

16

Explain it yourself

Explain to a product manager when you would run a feature on serverless functions and when you would not.

0 words