How AI coding agents work (and why "done" is not proof)
Monday, 9:40. Maya asked Claude Code to add a late check-out column to the housekeeping board used by 3,000 hotels. Four minutes later it replied: "Done. All tests pass." The board now crashes on load.
What really happens when you ask Claude Code for a change, and why "done" still needs checking.
- Problem
A product manager asks an AI coding agent to add a column to an internal dashboard. Four minutes later it says "Done, all tests pass". The page crashes on load.
- Why the simple system fails
Treating the agent like a compiler (it either works or errors) or like a colleague who has read the whole codebase. It is neither: it read a handful of files, guessed the rest, and its report is text, not a test result.
- Engineering concept
A coding agent is a language model in a loop with tools. It only knows what is in its context: your request, project instructions, and the files and command outputs it has pulled in. Its claims are only as good as the evidence it gathered.
- Technology
Claude Code's loop (read, edit, run, observe), CLAUDE.md project instructions, tool permissions, and asking for evidence: test output, file paths, the diff.
- Trade-offs
More autonomy and broader permissions make the agent faster and you less involved, but raise the cost of a wrong guess. More context helps until it crowds the window and costs time.
The problem
2 min- Maya · Product managerI asked it for a late check-out column on the housekeeping board, used by 3,000 hotels. Four minutes later: "Done. All tests pass."
- Maya · Product managerThe board crashes on load.
- You · Engineer at StayHubThe error says
getLateCheckouts is not exported from @stayhub/stays. - Lina · Senior engineer, your mentorGuess first.
The big idea
4 minThe agent only knows what it read. "Done" is a claim until you see proof.
So you have two levers: what goes into the context (rules, pointers to the right files) and what evidence comes out (command output, file and line, changed files). The rest of this module makes both precise.
How it works
5 minThink of a contractor who has renovated thousands of hotels but never seen yours. They work from the papers on their desk. Whatever they don't ask for, they fill in from experience: usually right, not always.
- 1A model in a loop
An is a language model running in a loop with tools in your project. Each turn it picks one tool call (read a file, edit it, run a command), sees the result, and decides the next step.
- 2Context is all it knows
Its context is everything it can see right now: your request, project instructions, files and outputs it pulled in. Maya's session read 7 of ~1,840 files. The real helper was in an eighth.
- 3Unread code is guessed
It needed late check-outs, so it imported the most plausible name,
getLateCheckouts. The real one,listLateCheckouts, was never read. - 4The summary is generated
No command ran, so no error ever came back. "All tests pass" was the likely ending of a finished task.
- 5CLAUDE.md: starting notes
is a file of project instructions loaded at the start of every session: test commands, where things live, what's off-limits, what "done" means. Sessions start empty; this doesn't.
- 6Evidence out
Turn each claim into something checkable: the test command and its output, the helper's file and line, the list of changed files, your own look at the page.
Watch a decision being made
4 minLina redoes Maya's change with context in and evidence out
Add the late check-out column again, so the agent reads before it writes and proves its result.
- 1Give it starting notes
Add a CLAUDE.md at the repo root: commands, where the board and stays helpers live, and what "done" means.
markdown## Commands - Test: pnpm test - Types: pnpm typecheck ## Where things are - Board: apps/ops/src/housekeeping/ - Stay helpers: packages/stays/src/ (check before writing one) ## Rules - "Done" = paste the test command and its output + changed filesWhy: Facts written once are in every session; facts said in chat vanish.
- 2Point, and make it read first
"Late check-outs already come from packages/stays. Before editing, find that function and tell me its file and line." Answer:
listLateCheckouts, checkout.ts, line 41.Why: Naming folders turns a 1,840-file search into a 3-file read, and an invented name can't produce a real file and line.
- 3Require evidence for done
It runs
pnpm testandpnpm typecheck, pastes the output (10 tests passed, no type errors) and lists the 3 changed files.Why: Now "tests pass" is a tool result, not a sentence.
- 4Check what users see
Maya opens the board and finds room 214 at the Lisbon test hotel, with its 14:00 check-out. It shows.
Why: Tests check what someone thought to test. The real screen is the last word.
In real life
3 minInline completion (e.g. GitHub Copilot suggestions)
- Suggests the next lines as you type
- Context is mostly the open file
- You read each suggestion before keeping it
Small steps; the human checks every line.
Agentic coding (e.g. Claude Code)
- Takes a whole task
- Explores, edits many files, runs commands
- Reports back at the end
Big steps; checking moves to evidence and review.
What goes wrong
3 minIt's generated text; it can describe tests that never ran.
Ask for the command and its output, the changed files, and where any helper lives.
It read a few files out of thousands and guessed the rest.
Point to folders; ask it to find existing code first.
The agent acts with your access; a guess becomes a real deletion.
Pre-approve evidence-producing commands; writes outside the repo wait for you.
Trade-offs
3 min| Permission rule | Fits when | Main risk |
|---|---|---|
| Ask before every command | First sessions; repos touching money or infrastructure | Approval fatigue: people stop reading |
| Pre-approve safe checks | Day-to-day: tests, type checks, lint, read-only git | A loose rule hides a risky command |
| Broad autonomy | Only in an isolated with nothing to lose | Guesses become real actions |
- Karim · CTOAllow everything; we review the result. Approvals kill the speed we bought.
- Omar · On-call SREWe lost 41,000 rows reviewing the result. Approve every command.
- Maya · Product managerHonestly, after twenty prompts I click "allow" without reading.
Your job as tech lead
4 minClaude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.
- What goes in CLAUDE.md: test and type-check commands, where key code lives, what's off-limits, what "done" means.
- Which commands run without asking (tests, lint, type checks, read-only git) and which always need you (databases, deploys, .env, network).
- What evidence you require before accepting: command output, helper file and line, changed files, your own look at the screen.
Add a "Late check-out" column to the housekeeping board in apps/ops/src/housekeeping. Late check-outs are already loaded in packages/stays: before editing, find that function and tell me its file and line. Don't write a new helper if one exists. Don't touch packages/payments or migrations. When finished, run pnpm test and pnpm typecheck, paste the last lines of each, and list every changed file.
- Scroll the session: a test command with real output, not just a sentence.
- Open the file and line of any helper it says it used.
- Compare the changed files with what you asked for; anything extra needs a reason.
- "All tests pass" with no test command in the session.
- It imports a function you can't find anywhere in the repo.
- It asks to write to a database or read .env for a UI-only task.
Check yourself
2 minNext: Maya's next request is one sentence long, and the agent will quietly decide everything she didn't say.
Mark it complete to update your skill map.
Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Visual diagram, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.
Technical deep dive
Redo Maya's change with good context and real evidence
Add the late check-out column to the housekeeping board again, this time so the agent reads before it writes and proves its result.
- 1Give it starting notes
Add a CLAUDE.md at the repo root: install and test commands, where the board and the stays helpers live, what needs permission, and what "done" means.
markdown## Commands - Test: pnpm test - Types: pnpm typecheck ## Where things are - Board: apps/ops/src/housekeeping/ - Stay helpers: packages/stays/src/ (check before writing one) ## Rules - "Done" = paste the test command and its output + changed filesWhy: The agent has no memory between sessions. Facts you write once are in every session's context; facts you say in chat vanish.
- 2Point to the right place
"Add a Late check-out column to the housekeeping board in apps/ops/src/housekeeping. Late check-outs already come from packages/stays."
Why: Naming folders turns a 1,840-file search into a 3-file read. Less guessing, less context spent.
- 3Make it read before it writes
"Before editing, find the existing function that lists late check-outs and tell me its file and line." It answers:
listLateCheckouts, packages/stays/src/checkout.ts, line 41.Why: Asking for the location forces a read and gives you something checkable. An invented name can't produce a real file and line.
- 4Require evidence for done
The agent runs
pnpm testandpnpm typecheck, pastes the output (10 tests passed, no type errors) and lists the 3 changed files.Why: Now "tests pass" is a tool result, not a sentence. The file list shows nothing unexpected was touched.
- 5Check what users see
Maya opens the board in the preview and finds room 214 at the Lisbon test hotel, which has a 14:00 check-out. It shows.
Why: Tests check what someone thought to test. The real behaviour, in the real screen, is the last word.
| Claim in the summary | Evidence to ask for |
|---|---|
| "All tests pass" | The command and its output |
| "I used the existing helper" | Its file and line |
| "Only the board changed" | git status or the diff |
| "It works" | Your own look at the real page |
Visual diagram
One agent session. The model never touches your files directly: every read, edit and command goes through a tool, and every result lands back in the context.
Analogy
- A contractor's first dayPapers on the deskA coding agentIts context
- A contractor's first dayAsking for a drawingA coding agentA tool call: read a file
- A contractor's first dayThe house rules binderA coding agentCLAUDE.md
- A contractor's first day"All done!" on the phoneA coding agentThe agent's summary
- A contractor's first dayThe inspector's reportA coding agentTest output you can see
Where the analogy breaks: a human contractor remembers your hotel tomorrow. The agent starts each session fresh, except for what you wrote down in CLAUDE.md.
Why it matters
Too much trust
- "It said done, ship it"
- Invented functions reach users
- Untested code goes live
Fast, then broken.
Too little trust
- Check every single character
- Rewrite what it did
- Lose all the speed
Safe, but slow.
Calibrated
- Feed it the right files
- Ask for the test output
- Check what users see
Fast and trustworthy.
Check your understanding
- 1.The result is added to the context for the next turn
- 2.The tool performs the action
- 3.The model proposes the next action (e.g. open a file, run the tests)
- 4.If the action isn't pre-approved, you are asked to allow it
- 5.Your request and project instructions are in the context
Practice ladder
Read a session log
Debugging Warm-upHere is the tool activity from a session where Maya asked for "show the guest's arrival time on the housekeeping board". The final message says: "Done! Arrival time is shown and all tests pass."
| # | Tool call | Result |
|---|---|---|
| 1 | Read apps/ops/src/housekeeping/BoardTable.tsx | 180 lines |
| 2 | Read apps/ops/src/housekeeping/types.ts | 40 lines |
| 3 | Edit BoardTable.tsx | +6 −1 |
| 4 | Edit types.ts | +1 |
For each claim in the final message, say whether the log supports it, and what you'd ask for.
Finish the CLAUDE.md
Hands-on PracticeLina started a CLAUDE.md for StayHub's internal-tools repo. Fill in the three missing parts.
# StayHub internal tools
## Commands
- Install: pnpm install (never npm)
- Test: ___ (1)
- Types: pnpm typecheck
## Where things are
- Housekeeping board: apps/ops/src/housekeeping/
- Stay helpers: ___ (2)
## Rules
- Ask before touching migrations or packages/payments/
- Never read or print .env files
- "Done" means: ___ (3)A request for the reviews tool
Hands-on PracticeKarim wants the guest-reviews moderation tool to hide reviews containing phone numbers. The repo has moderation rules in src/moderation/, tests with pnpm test, and a scripts/ folder that touches the production reviews database. There is no CLAUDE.md.
Write (a) the 4–6 lines you'd add to CLAUDE.md, and (b) the request you'd give Claude Code.
Background agent for the whole team?
Decision StretchKarim proposes letting Claude Code work through the internal-tools backlog overnight with every command allowed, "so we wake up to finished features". The team's laptops have staging database credentials in .env files.
Write a short decision: "We chose X over Y because…, accepting the risk of…", including what setup would make overnight work acceptable.
Explain it yourself
A teammate says: "Claude Code is basically a senior engineer who has read our whole codebase. If it says done, it's done." Explain what's wrong with that picture and what you'd do instead.