Episode 113 · 41 tool calls to Porto

Planning, loops and knowing when to stop

Wednesday, 15:20: a guest asked for a quiet hotel in Lisbon under €150. Twenty minutes, 41 tool calls and €3.10 of tokens later, the agent is still searching, and it's now comparing hotels in Porto.

An agent is a loop around a model. The hard part isn't starting the loop, it's planning well and knowing when to stop.

30 min advancedReAct loop: reason, act, observePlanning: plan-and-execute vs step-by-stepStep limits and stopping conditionsAgent failure modes: loops, drift, compounding errorsReflection and self-critiqueThe agent loop

Problem
A travel agent bot is asked to find a quiet hotel in Lisbon under €150. Twenty minutes and 41 tool calls later it is still searching, has spent €3 in tokens, and is now comparing hotels in Porto.
Why the simple system fails
A naive loop (“call the model until it says done”) has no budget, no notion of progress, and no defence against the model repeating itself or slowly wandering away from the goal. Each step's small error rate compounds over long runs.
Engineering concept
Make the loop an engineered control system: explicit state, a plan or progress checklist, hard budgets, stopping conditions checked in code, loop and drift detection, and reflection only where there is something checkable to reflect against.
Technology
ReAct-style tool loops, plan-and-execute, step and token budgets, repeated-call detection, goal re-anchoring, self-critique against criteria, traces of every step.
Trade-offs
More autonomy handles messier tasks but costs tokens, latency and predictability. Planning up front is cheaper and auditable but brittle when reality differs; reactive loops adapt but drift. Every guard rail also stops some legitimate runs.

The problem

2 min

Wednesday, 15:20. A new "flexible search" mode raised the agent's budget to 60 steps.

Omar · On-call SRE
Run #58213: 41 tool calls, 20 minutes, €3.10. Goal: quiet hotel in Lisbon under €150. Its last searches: Porto.
Maya · Product manager
The guest gave up after 12 minutes. A normal run costs €0.12.
You · Engineer at StayHub
The model got confused. A smarter model?
Lina · Senior engineer, your mentor
Guess first. Then read the trace with me.

The big idea

4 min

How a loop wanders

The big idea

The hard part of an agent loop is stopping: on a budget, on repeats, on no progress, and on the original goal. Only code can say stop.

If each step is right 95% of the time, a 10-step run is fully right only about 60% of the time (0.95¹⁰). Long loops don't need a slightly better model; they need checkpoints and early stops.

How it works

5 min

Every agent runs a ReAct loop (): reason about the next step, act with a tool, observe the result, repeat. Nothing in it says when to give up. Here is what run #58213 needed:

A loop that ends

1
A plan with "done"
Plan-and-execute (): one call writes the steps and the success criterion ("a quote in Lisbon, 21–23 Nov, ≤ €150, confirmed"), then re-plans only when a step fails.
2
A budget
A step budget () in code: 12 steps, €0.30, 60 seconds, checked before every model call.
3
Stuck detection
Same tool and arguments 3 times, or 4 steps with no new candidate: stop repeating. Compact tool results (hotel, price, rating) keep the context small.
4
A pinned goal
City, dates and price checked in code on every call. That stops goal drift (): slowly pursuing something else, like Porto.
5
Verified "done"
The model can say "done"; the runtime checks real state (a booking id that exists) before the final message.
6
An honest exit
Stuck or out of budget: say what was found and ask ("Nothing quiet under €150; €165, or another weekend?"), or hand over with the trace.

Watch a decision being made

4 min

Worked example

Lina rebuilds the loop so it ends

Make the agent finish, or stop honestly, on hard requests like run #58213, at no more than €0.30 a run.

1Define done first
"What does done look like? A quote in Lisbon, 21–23 Nov, ≤ €150, good quiet reviews, confirmed by the guest. The planner writes that down."
Why: Without a success criterion, no result is ever good enough and the loop keeps searching.

2Put the stop rules in the runtime

"Budget, repeats, no progress, goal check, before every model call."

if (step >= 12 || spent.eur >= 0.30) return stop("budget");
if (repeats(state.calls, 3)) return stop("repeating");
if (noProgress(state, 4)) return stop("no-progress");
if (violatesGoal(nextCall, goal)) return stop("drift");

Why: €3.10 runs become impossible by construction, and widening the search becomes the guest's choice, not the model's.

3Make every stop useful
"Out of budget? Tell her what we found and ask: Hotel Alfama at €165, or 28–30 Nov at €140?"
Why: A bounded failure that asks a good question often becomes a booking; a silent timeout never does.
4Measure it
Median steps 9 → 6, p95 38 → 11, cap hits 2%, cost per run €0.12 → €0.09.
Why: The fix is proven on real runs, not on one replayed incident.

In real life

3 min

Open-ended loop vs bounded task

AutoGPT (2023): an open-ended loop

A viral open-source project: give a model a goal and let it loop with tools
Widely reported to get stuck, repeat itself and burn tokens on simple goals

The cost of a loop with no plan and weak stopping.

ChatGPT Deep Research (2025): clarify, then bound

Asks clarifying questions before it starts
Then runs a multi-step research task and returns a cited report
Scope agreed first; the run ends with a defined deliverable

A clear goal and a defined end make long runs usable.

What goes wrong

3 min

Letting the model decide when the loop ends

It loops forever, or declares success after a failure.

Instead

Stop rules and success checks in the runtime, against real state.

"Don't loop" in the prompt as the only guard

Prompt rules are suggestions; under failing tools they're ignored.

Instead

Budgets, repeat and no-progress detection in code.

A "review your answer" step after every call

Reflection without new information mostly rubber-stamps, at double the tokens.

Instead

Check computable criteria (city, price, booking id) in code.

Trade-offs

3 min

Loop design	Choose when	You pay
Step-by-step (ReAct)	Short, exploratory tasks	Wanders more; hard to audit
Plan-and-execute	Clear goal and success criteria, like find-and-book	A planner call; re-plans when reality differs
Workflow (no loop)	The steps are always the same	Only the paths you designed

Monday, 10:00. Plan-first for every run?

Omar · On-call SRE
Plan-and-execute for every run. Auditable, cheaper, and the plan doubles as a checklist.
Maya · Product manager
Plans break. A guest changes her mind mid-chat and the plan is wrong. Step-by-step just adapts.
Omar · On-call SRE
Then re-plan when a step fails or the guest changes something.

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own

The hard limits on every run, in the runtime: 12 steps, 60 s, €0.30. A 60-step "flexible" mode is your decision, not the agent's.
The stop rules: 3 near-identical calls, no new candidate for 4 steps, success criteria met.
What the guest sees when a run stops without success: an honest status and options, never a made-up result.

Tell Claude Code

A find-and-book run made 41 tool calls (€3.10) and drifted from Lisbon to Porto, repeating near-identical searches. In the agent runtime, add: a hard budget of 12 steps, €0.30 and 60 s; a repeat guard after 3 near-identical calls; a no-progress rule; the guest's constraints checked on every call; compact tool results. Verify city, dates and price in code before showing a result, and never say "booked" unless the booking exists. Plan first, then replay the last 200 runs and report median and p95 steps and cost.

Check before you accept

Each limit lives in runtime code that ends the loop, not in the system prompt.
The incident run, replayed, stops by step 12 or at the repeat guard, with an honest message.
A forced create_booking failure gives "didn't go through", never "Booked!".

Red flags

Raising the step budget so long runs "can finish".
"Don't repeat searches" in the prompt, with no detection in code.
The model's final text as the only signal that a booking succeeded.

Check yourself

2 min

Q1/3Reasoning

Each step of an agent run is correct 97% of the time, independently. Roughly what percentage of 20-step runs are correct at every step?

How confident are you?

Next episode: the guest-service agent can rebook, refund and email. One afternoon it refunds €4,800 to a guest who only asked "what's your refund policy?".

Finished?

Mark it complete to update your skill map.

Next: Human in the loop: approvals, thresholds and escalation

Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.

Technical deep dive

Plan, budget, stop

Worked example

You rebuild the loop so it ends

Make the find-and-book agent finish (or stop honestly) on hard requests like run #58213, at no more than €0.30 a run. Lina asks one question per step.

1Plan first
Lina: "What does done look like?" You: one planner call writes the steps and the success criterion: "a quote in Lisbon, 21–23 Nov, ≤ €150/night, with good quiet reviews, confirmed by the guest".
Why: Without a success criterion, no result is ever good enough, and the loop keeps searching.
2Pin the constraints in code
City, dates and max price from the plan become a check on every tool call. A call outside them is blocked and becomes a question to the guest.
ts
```
function violatesGoal(call: ToolCall, g: Goal) {
  const a = call.args;
  return (a.city && a.city !== g.city) ||
         (a.max_price && a.max_price > g.maxPrice) ||
         (a.dates && !sameDates(a.dates, g.dates));
}
```
Why: This is what stops Lisbon becoming Porto. Widening the search is the guest's choice, not the model's.
3Set the budgets
12 steps, €0.30 of tokens, 60 seconds, checked before every model call.
Why: Budgets turn a runaway into a bounded failure. €3.10 runs become impossible by construction.
4Detect stuck loops
Same tool and arguments 3×, or 4 steps without a new candidate hotel: stop repeating. Compact tool results to key fields (hotel, price, rating) instead of full JSON.
Why: Repeats and no-progress are the signatures of a stuck loop; compact results keep the goal near the top of the context.
5Exit honestly
On any stop: tell the guest what was found and ask ("Nothing quiet under €150 those dates. Hotel Alfama at €165, or 28–30 Nov at €140?") or hand over with the trace. Lina: "Good. And measure it." Median steps 9 → 6, p95 38 → 11, cap hits 2%.
Why: A bounded failure that asks a good question often becomes a booking; a silent timeout never does.

Stop rule	Detects	On trigger
Step / token / time / money budget	Runaway runs	Stop, summarise what was found, ask or hand over
Repeated call (same tool + args × 3)	Stuck loops	Stop repeating; change approach or ask the guest
No progress (N steps, nothing new)	Wandering	Re-plan once, then ask or hand over
Goal check on every call	Drift (wrong city, price, dates)	Block the call; ask before widening
Verified success	False "done"	Check real state (booking exists) before the final message

Every rule lives in the runtime, not the prompt.

Interactive simulation

Visual diagram

Guest

Agent runtime

LLM (plan / reason)

Tools

Run state

Stop rules + checks

Human / user

Click a component

A controlled agent loop. The model proposes; the runtime decides whether the loop continues.

Analogy

An employee on a research taskAn agent loop

An employee on a research taskTry something, look, decide what's nextAn agent loopThe ReAct loop
An employee on a research taskA to-do list the manager can readAn agent loopPlan-and-execute
An employee on a research taskExpense limit and deadlineAn agent loopStep, time and cost budgets
An employee on a research task"Stop when booked or after 10 hotels"An agent loopStopping conditions in code
An employee on a research taskRe-reading the brief before sendingAn agent loopReflection against clear criteria
An employee on a research taskA great report on the wrong questionAn agent loopGoal drift

Where the analogy breaks: an employee feels tired after the 30th failed try. An agent never does. Without limits set in code, it will happily keep going.

Why it matters

An agent works in a loop: it asks your code to run a tool (search hotels, check availability), reads the result, and decides the next step. Its bugs rarely crash. They look like this.

Runaway cost. The agent retries the same failing search 30 times. Each loop resends a growing history, so the bill climbs fast.

Silent failure. It hits a wall, then describes what it would have done as if it had done it.

Drift. After 15 steps, the context is full of tool output. The guest's "under 150 euros, quiet" has faded from focus. That's .

Slowness. Each step is a model call plus a tool call. Ten steps can mean a minute of waiting.

The loop's design, with plans, budgets, stop rules and checks, decides whether your agent is a product or a demo.

Check your understanding

Q1/3Reasoning

Adding a “review your answer and improve it” step after every model call reliably improves agent quality.

How confident are you?

Practice ladder

Step 1 · Trace

Where do the stop rules fire?

Debugging Warm-up

Goal: quiet hotel in Lisbon, 21–23 Nov, ≤ €150/night. Stop rules: budget 12 steps / €0.30; same tool + args 3× → stop repeating; 4 steps without a new candidate hotel → no progress; any call outside city/dates/price → blocked.

A trace (cumulative cost in brackets): 1. search_hotels(Lisbon, 21–23 Nov, ≤150, quiet) → 0 results (€0.02) 2. search_hotels(Lisbon, 21–23 Nov, ≤150) → 6 hotels (€0.04) 3. get_availability(h1) → full (€0.05) 4. get_availability(h2) → full (€0.06) 5. get_availability(h3) → full (€0.08) 6. get_availability(h4) → full (€0.09) 7. search_hotels(Lisbon, 21–23 Nov, ≤150) → same 6 (€0.11) 8. search_hotels(Lisbon, 21–23 Nov, ≤150) → same 6 (€0.13) 9. proposed: search_hotels(Lisbon, 21–23 Nov, ≤170)

Trace which rule fires first and when, and what the runtime should tell the guest.

Step 2 · Complete

Write the success check

Code Practice

Complete verifyDone so the runtime only sends a "booked" message when a booking really exists and matches the goal; otherwise it returns an honest message.

async function verifyDone(final: FinalAnswer, goal: Goal, state: RunState) {
  const bookingId = state.lastBookingId;          // set only when create_booking returned ok
  // TODO 1: no booking id → not done
  // TODO 2: fetch the booking from our system and check city, dates, price
  // TODO 3: return the message the guest should see
}

Step 3 · Solve

Budgets for the group-booking agent

Design Practice

Sales wants an agent for group bookings (20–80 rooms for weddings and conferences): it checks several hotels, compares group rates, reads group policies and drafts an offer for a salesperson. A good run takes 15–25 steps; a salesperson waits up to 10 minutes. The offer is reviewed by a person before it's sent.

Design the plan, the budgets and stop rules, and what happens when a run stops early.

Step 4 · Decide

Self-critique before the final answer?

Decision Stretch

A teammate proposes adding a self-critique step: before every final answer, the model reviews its own answer and fixes problems. It adds one call (≈ €0.02 and ~2 s) to every find-and-book run (≈3,200 a day).

Facts from a test on 200 runs: the critique changed 9% of answers; on review, half of those changes were improvements and half made things worse or were neutral. The errors that mattered most were wrong prices and dates, which tools already know.

Write a short decision: "We chose X over Y because…, accepting the risk of…".

Explain it yourself

Explain how you would make an agent loop safe to run in production: how it plans, how it stops, and how you'd know it's misbehaving.

0 words