Episode 159 · Who reviewed this?

When to ask a human: knowing your limits

Tuesday, 11:20. Claude Code's pull request makes cancelled bookings refund automatically: 40 lines, 6 tests, all green. StayHub refunds about 1,100 cancellations a day. The PR touches the payment service. Karim asks one question: "Who reviewed this?"

The agent makes every change feel equally easy. You decide which ones need an expert's eyes.

30 min intermediateTiering changes by risk (blast radius, reversibility, expertise)One-way-door changesSignals you are past your competence boundaryA good handoff to a human expertBlast radius and cohort comparisonPull requests, code review & small batches

Problem
A 40-line, all-green pull request from Claude Code makes cancelled bookings refund automatically. It touches the payment service. The person who would merge it can't say what happens if a cancellation event arrives twice.
Why the simple system fails
An agent makes every change look equally easy: same tone, same green tests, same tidy diff. So people review a payment change like a button colour, and one-way doors (money, deletions, auth, data migrations) get walked through without anyone who could judge them.
Engineering concept
Rate each change on three dials (blast radius, reversibility, whether you can verify it yourself), know the signals that you're past your competence, and bring in an expert early with a precise question, a small diff and what you already checked.
Technology
Risk tiers, one-way vs two-way doors, CODEOWNERS and required reviews, CLAUDE.md stop rules and hooks on protected paths, handoff templates, idempotency keys as the classic example of expert knowledge.
Trade-offs
Asking costs an expert's time and delays you; not asking is free until a one-way door goes wrong. The aim isn't to ask about everything, but to spend expert attention where mistakes can't be undone.

The problem

2 min

Tuesday, 11:20. Support refunds ~1,100 cancellations a day by hand. Your PR #2817 makes it automatic.

You · Engineer at StayHub
It's ready: 40 lines, 6 tests, all green.
Karim · CTO
It moves money. Who reviewed this?
You · Engineer at StayHub
I did. I read the diff; it's clean.
Lina · Senior engineer, your mentor
You lead. Is your review enough?

The big idea

4 min

Forty lines, one question

The big idea

Rate every change by blast radius, reversibility and whether you can verify it, not by its size; past your competence boundary, hand it off.

With an agent, a refund change costs the same five minutes as a label change. A good drill makes hanging a shelf and drilling into the fuse box equally easy; the tool doesn't change the risk.

How it works

5 min

Three dials, then a decision

1
Blast radius
: who is affected, and how much, if it's wrong. PR #2817: every refunded cancellation, ~1,100 a day, ~150 € each. High.
2
Reversibility
A is expensive or impossible to undo: money out, data deleted, mass emails. Refunds paid must be clawed back one by one. No.
3
Can I verify it?
Your competence boundary is the line past which you can't explain or check a change, so your approval checks nothing. Duplicates? Vouchers? No.
4
Hand off precisely
A handoff is short: goal, change, what you checked, one question. A precise question takes an expert ten minutes; "can you look at my PR?" waits a week.
5
Ask at the plan
Show the owner the plan before code: five minutes on a plan beats an hour on a finished diff, and sunk cost never pushes you to merge.
6
Make it automatic
makes risky paths request their owner; a CLAUDE.md stop rule makes the agent explain and wait.

Watch a decision being made

4 min

Worked example

You get PR #2817 safely to production; Lina only checks

Ship automatic refunds without walking through a one-way door blind.

1Rate it in the PR
Blast radius high, reversible no, can I verify no. Tier: expert before merge.
Why: Written dials make the decision visible to Karim, and to you next time.
2Sharpen the question with the agent
"Explain how a cancellation reaches this code; what happens if the event arrives twice, late, or for a voucher?" It says the event "should" be delivered once.
Why: The agent maps the code well; its "should" shows where knowledge runs out.
3Hand off with one question
To Inès (payments owner): spec, unmerged PR, what you checked, and: "If the event is delivered twice, do we refund twice? I see no idempotency key."
Why: Ten minutes of her time instead of an afternoon.
4Turn the answer into a test and a rule
Inès: the queue delivers at least once; last month a restart redelivered ~3,000 events. Use the cancellation id as the ; test two deliveries → one refund. Add CODEOWNERS for src/payments.
Why: Her ten minutes prevented ~3,000 double refunds (~450,000 €); the test and CODEOWNERS make it permanent.

In real life

3 min

Approve before, or revert after

Chromium: OWNERS files

Owners of each touched directory must approve
Experts in the path to merge
Browser code reaches billions

Pre-approval for one-way doors.

Wikipedia: edit, then revert

Almost anyone edits immediately
Many watchers, instant revert
Mistakes are cheap to undo

Speed for two-way doors.

What goes wrong

3 min

Judging risk by diff size or effort

A 40-line payment change looks like a 40-line label change.

Instead

Rate blast radius, reversibility and whether you can verify it.

Approving what you can't explain

The review checks nothing.

Instead

If you can't say how you'd know it's wrong, hand it off.

Running deletions without a dry run

A wrong definition deletes real data; restores lose recent changes.

Instead

Dry-run report, owner review, rehearsed restore, then run.

Trade-offs

3 min

Review rule	Fits when	You pay
Expert review on everything	Briefly while learning, or regulated code	Bottleneck; rushed reviews
Risk-tiered, enforced by CODEOWNERS	Most teams, most of the time	Honest rating; misclassification
Self-merge with green tests	Solo prototypes, no money or personal data	One-way doors unreviewed

Wednesday, 10:00. Review rules after PR #2817.

Karim · CTO
Every PR written with Claude Code is reviewed by an engineer. No exceptions.
Maya · Product manager
Then a handful of engineers become everyone's bottleneck. We built these tools so non-engineers could ship small things.
Karim · CTO
PR #2817 looked small.

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own

The tier of every change, from the three dials, written in the PR.
Which paths and actions are one-way doors (payments, auth, migrations, deletions, mass messages, legal text) and who owns each.
When to stop and hand off: you can't explain it, can't say how you'd know it's wrong, or you're accepting to be done.

Tell Claude Code

Plan automatic refunds for flexible bookings cancelled 48+ hours before check-in (docs/specs/auto-refund.md). Don't write code: this touches src/payments, a stop-and-ask area. Write the plan in docs/plans/auto-refund.md explaining how a cancellation reaches the refund code, what happens if the event arrives twice or late, and for voucher payments. Say "I checked" or "I don't know", not "should".

Check before you accept

Try to explain the change and how you'd know it's wrong; if you can't, hand it off.
For one-way doors, the owner saw the plan and their answer became a test.
CODEOWNERS covers every risky path the change touches.

Red flags

A small, green diff in src/payments or migrations nobody from that team has seen.
The agent says something "should" be fine instead of what it checked.
You notice you're approving because it's late and everything is green.

Check yourself

2 min

Q1/3Recall

Which three dials decide whether a change needs an expert?

How confident are you?

Last episode of the season: one real feature, from Maya's request to guests using it. You direct Claude Code; Lina only checks.

Finished?

Mark it complete to update your skill map.

Next: Capstone: ship a real feature with Claude Code

Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.

Technical deep dive

Ask well, ask early

Worked example

Getting PR #2817 safely to production (Lina only checks)

Ship automatic refunds without walking through a one-way door blind: rate the change, hand off to the right expert with a precise question, fix what they find, and make sure the next payments change reaches them automatically.

1Rate it in the PR
Blast radius high (every refund), reversible no (money out), can I verify no (duplicates? partial refunds? vouchers?). Tier: expert before merge.
Why: Writing the dials down makes the decision visible to Karim and to you next time, instead of a feeling.
2Prepare the question with the agent
Ask Claude Code: "Explain in plain language how a cancellation reaches this code, and list what happens if the event arrives twice, late, or for a voucher payment." It says the event comes from the bookings queue and that it "should" be delivered once.
Why: The agent is great at mapping the code; its "should" tells you where your knowledge, and its, runs out.
3Hand off precisely
Message Inès (payments owner): goal and spec, PR link (not merged), what you checked, and one question: "If the cancellation event is delivered twice, do we refund twice? I don't see an idempotency key."
Why: A precise question takes an expert ten minutes; "can you look at my PR?" takes an afternoon and often waits a week.
4Fix what the expert finds
Inès: the queue delivers at least once; last month a restart redelivered ~3,000 events. Ask Claude to use the cancellation id as the and add a test that sends the same event twice and expects one refund.
Why: Her ten minutes prevented ~3,000 double refunds (~450,000 €). The test makes her knowledge permanent.
5Make the line automatic
Add for src/payments, src/auth, db/migrations, infra, and a CLAUDE.md stop rule: before changing those paths, explain the risk and wait.
Why: Next time, nobody has to remember to ask: the merge button and the agent both raise it.
6Next time, ask at the plan
For the next payments feature, show Inès Claude's plan before any code is written.
Why: Five minutes on a plan beats an hour on a finished diff, and sunk cost never pushes you to merge.

markdown

## Stop and ask
Before changing src/payments, src/auth, db/migrations or infra, or writing code that
deletes data, messages many guests or moves money: stop, explain the change and its
risks in plain language, and wait for my go-ahead. Don't say "should"; say what you checked.

The stop rule added to CLAUDE.md.

Analogy

DIY at homeDirecting an agent

DIY at homePainting a roomDirecting an agentChanging a label or layout
DIY at homeRewiring the fuse boxDirecting an agentChanging payments or logins
DIY at homeThe drill makes both easyDirecting an agentThe agent makes both easy
DIY at homeCall the electricianDirecting an agentAsk the payments owner
DIY at home"Can this circuit take an oven?"Directing an agentOne precise question

Where it breaks: a bad paint job is visible right away. A payment mistake can look fine for weeks, so you can't rely on seeing it.

Why it matters

Some mistakes are cheap. A wrong label gets fixed in a minute.

Others aren't. Money paid twice has to be asked back from guests, one awkward email at a time. Deleted data only comes back from a backup, and everything since is lost. Security mistakes are usually found by someone else, much later.

The agent writes code in all of these with the same confidence. It doesn't know that your booking system sometimes sends the same message twice, or what your contracts say. A colleague does, and asking costs ten minutes.

Check your understanding

Q1/3Decision

Which of these are one-way doors? (Select all that apply.)

Select all that apply.

How confident are you?

Practice ladder

Step 1 · Trace

Rate five changes on the three dials

Decision Warm-up

For each change Claude Code proposes, rate blast radius (low/medium/high), reversible (yes/no), and whether you can verify it (yes/no), then pick a tier: self-review, peer review, or expert before build.

Add a "copy booking reference" button in the guest app.
Change how session cookies are configured "to fix a login bug on Safari".
Send a "we've updated our privacy policy" email to all 300,000 guests.
Add a column "VIP" to the internal hotel report.
Recalculate and overwrite stored loyalty point balances after a formula fix.

Step 2 · Complete

Finish the handoff to Inès

Scenario Practice

Complete the handoff for a new payments change: Claude Code has a plan (no code yet) for partial refunds when a guest shortens a stay.

Goal: ___
Change: ___
What I checked: ___
What I can't judge / question: ___
Timing: ___

Step 3 · Solve

The Safari login fix

Decision Practice

Guests on Safari are sometimes logged out after paying (they come back from the payment page logged out). Claude Code proposes changing the session cookie settings and "relaxing" a security attribute, and says this is "a common fix". You don't know what the attribute does.

Decide what to do, what you ask Claude Code, and write the handoff to the security owner.

Step 4 · Decide

GDPR deletion by Friday, expert on holiday

Decision Stretch

A guest has asked StayHub to delete all their personal data, and the legal deadline is close. Maya wants an automated "delete my data" feature by Friday. Claude Code has a plan: delete the guest's row and all related rows across 14 tables. The data owner (platform team lead) is on holiday until next Wednesday. Karim says: "Just ship it; it's only deletes."

Write your decision: "We chose X over Y because…, accepting the risk of…".

Explain it yourself

Maya asks: "If Claude Code can write payment code and the tests pass, why do we still need Karim's team to look at it?" Answer her.

0 words