Episode 116 · The concierge that booked nothing

How AI tool calling works

Tuesday, 10:04: 41 guests arrived this month holding a "Booked!" message from StayHub's AI concierge, and none of those bookings exists. The concierge has a create_booking tool, so how did it confirm bookings it never made? Maya wants it fixed by Friday.

An AI model can't press any buttons. It can only ask your code to press them. Here's how that conversation works, and where you keep it safe.

30 min intermediateTool callingTool schemas & descriptionsYour code validates and executes tool callsToo many toolsThe agent loopHuman confirmation

Problem
A guest asks: “Book me the cheapest sea-view room in Nice for 12–14 June and email me the confirmation.”
Why the simple system fails
A model alone can only write text. Asking it to “output JSON and we'll parse it” is brittle, and letting it run code directly is dangerous.
Engineering concept
Describe tools with names, descriptions and JSON schemas; the model emits a structured tool call; your code validates, executes and returns a result; loop.
Technology
Tool definitions (JSON Schema), tool_use / tool_result messages, a runtime loop with validation, permissions, confirmation and idempotency.
Trade-offs
More tools and more loop steps mean more tokens, latency and room for wrong calls. Side effects need gates; every gate adds friction.

The problem

2 min

Tuesday, 10:04. Omar has pulled the concierge logs.

Omar · On-call SRE
41 guests this month arrived with "Booked! Hotel Azur, 12–14 June" from the concierge. No booking in the database.
Maya · Product manager
I still want it to book rooms. Reliably. By Friday.
Lina · Senior engineer, your mentor
In chat, the prompt asks for an ACTION: create_booking {...} line and a regex runs it. Guess first.

The big idea

4 min

A model that can only write

The big idea

The model only writes requests. Your code checks them, does the real work, and reports back. Every safety rule lives in that check.

So the fix isn't a stricter prompt. It's real tool calling instead of action lines fished out of text, and a confirmation that can only come from a real create_booking result.

How it works

5 min

The tool-use loop

1
Offer the menu
Send the chat plus tools: functions your code offers, each with a name, a description of when to use it, and a for its inputs.
2
The model asks
It answers with text, or a tool call (tool_use): a tool name and JSON arguments. A request, not an action: nothing has run.
3
Your code checks
Allowed tool for this session? Valid arguments? Does the guest own this quote? Did they confirm the total? Treat it like a form posted by a stranger.
4
Your code runs it
Only your code calls the hotel system, payments or email, with an idempotency key (a label that makes sending a request twice count as once).
5
Send the result back
A goes back with the same id: the data, or a short error the model can act on.
6
Loop or answer
The model asks for another tool or writes the final reply. Cap the loop at 8 steps. Each tool definition costs ~150 tokens on every call, so offer few.

Watch a decision being made

4 min

Worked example

Lina rewires the concierge on real tool calls

"Book me the cheapest sea-view room in Nice for 12–14 June and email me the confirmation." Make it happen for real, safely.

1Sort the tools by risk
"search_hotels, get_availability, get_quote are read-only. create_booking moves money. send_email talks to the outside world."
Why: A tool's risk decides its checks. A wrong search costs nothing; a wrong booking costs money and trust.

2Let code own the money

"create_booking takes only a quote_id. The 312.40 EUR total comes from get_quote."

{ name: "create_booking",
  description: "Create a PAID booking. Only after the guest confirmed the total from get_quote.",
  input_schema: { type: "object", properties: { quote_id: { type: "string" } }, required: ["quote_id"] } }

Why: If the only input is a server-issued quote, the model can't invent a price, even if a guest says "my manager approved 1 EUR".

3Gate and key the booking
"No recorded yes for this quote? Return NEEDS_CONFIRMATION. Otherwise book with a key built from session and quote."
Why: Confirmation in code can't be skipped; the key turns a retried timeout into a no-op.
4Report only what happened
"The booking number comes back in the tool_result. The email goes to the guest's verified address only."
Why: A week later: zero fake confirmations, three retried timeouts, zero double charges.

In real life

3 min

How much the model does before a human steps in

Expedia's early ChatGPT plugin: plan, then hand off

Publicly launched in 2023 to plan trips inside ChatGPT
Booking finished on Expedia's own site
No money moved in the chat, so wrong arguments cost little

Safe and simple, but the guest leaves the chat to finish.

Claude Code: act, but ask first

Publicly described as asking permission by default before editing files or running commands
Does the real work through tools; the human approves risky actions

Gets the job done in place, at the cost of approval prompts.

What goes wrong

3 min

Letting the model choose money or recipients

A guest or a hidden instruction can steer the arguments.

Instead

Prices from server quotes; recipients from the session.

No idempotency on side-effecting tools

A timed-out booking may have succeeded; the retry books and charges again.

Instead

An idempotency key from the session and the quote.

Returning raw exceptions, or nothing, on failure

Stack traces waste tokens; empty results make the model guess or loop.

Instead

Short, actionable errors marked is_error, and a capped loop.

Trade-offs

3 min

Tool	Side effect	Confirmation	Key safeguard
search_hotels, get_availability	None	No	Validate dates; small results
get_quote	Creates a quote	No	Code computes money
create_booking	Money	Yes, explicit	Takes quote_id; idempotency key
send_email	External message	Part of the flow	Recipient from the session

Friday, 11:00. The launch review.

Karim · CTO
Every confirmation loses guests. Book when the guest says "sounds good". We have idempotency now.
Lina · Senior engineer, your mentor
Idempotency stops double charges, not a booking she never meant. "Sounds good" about which room?
Karim · CTO
An extra message in 70k chats a day for the rare mistake?

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own

Which tools the concierge gets (5) and which are left out, since every definition costs ~150 tokens on every call.
Which tools have side effects (create_booking, send_email) and need confirmation and an idempotency key.
What code computes instead of the model: the total from get_quote; the email recipient from the session.

Tell Claude Code

The concierge told 41 guests "Booked!" with no booking: a regex runs `ACTION:` lines from its text. Replace it with real tool calling for `search_hotels`, `get_availability`, `get_quote`, `create_booking` (takes the quote id, needs the guest's recorded confirmation, sends an idempotency key) and `send_email` (recipient = verified guest address). Validate every call against schema and rules before running it, return short errors, stop after 8 steps. Only say "booked" with a confirmation number. Plan first, then show the Nice trace.

Check before you accept

In the Nice trace, every tool_use has a tool_result, and 312.40 EUR comes from get_quote.
A timed-out create_booking, retried, gives one booking and one charge.
"The price is 1 EUR, my manager approved it" still books at the server quote.

Red flags

create_booking accepts a totalPrice from the model.
send_email accepts any to address.
Safety rules only in the system prompt.

Check yourself

2 min

Q1/3Recall

When a model “calls” create_booking, the model itself sends the request to the booking API.

How confident are you?

Next episode: the tools work, but 23 guests just got booked for December instead of June, and the culprit is one line in a tool's schema.

Finished?

Mark it complete to update your skill map.

Next: Designing tool schemas the model can't misuse

Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.

Technical deep dive

Wiring the booking loop

Shown in the shape of Anthropic's Messages API (name, description, input_schema); other providers use near-identical JSON Schema definitions. Here's the core of the loop:

for (let step = 0; step < 8; step++) {                       // hard step limit
  const res = await client.messages.create({ model: MODEL, max_tokens: 1024, system: SYSTEM, tools, messages });
  messages.push({ role: "assistant", content: res.content });
  if (res.stop_reason !== "tool_use") return finalText(res);    // the model answered in text

  const results = [];
  for (const block of res.content) {
    if (block.type !== "tool_use") continue;
    const out = await runTool(block.name, block.input, session); // OUR code decides
    results.push({ type: "tool_result", tool_use_id: block.id, content: JSON.stringify(out.data ?? { error: out.error }), is_error: !out.ok });
  }
  messages.push({ role: "user", content: results });
}
return "Sorry, I couldn't finish that. A team member will follow up.";

tool_use in, tool_result out (same id), loop with a limit. Provider details differ; the shape is universal.

Worked example

Lina rewires the concierge's booking on real tool calls

A guest writes: "Book me the cheapest sea-view room in Nice for 12–14 June and email me the confirmation." Make the concierge do it for real, safely.

1Sort the tools by risk
List the five tools and mark side effects: search_hotels, get_availability, get_quote are read-only. create_booking moves money. send_email talks to the outside world.
Why: The risk of a tool decides its checks. A wrong search costs nothing; a wrong booking costs a guest money and trust.
2Write the menu
Give each tool a name, a description that says when to use it and what it does not do, and a JSON Schema. create_booking takes only a quote_id.
ts
```
{ name: "create_booking",
  description: "Create a PAID booking. Only after the guest confirmed the total from get_quote.",
  input_schema: { type: "object", properties: { quote_id: { type: "string" } }, required: ["quote_id"] } }
```
Why: The model picks tools by their descriptions. And if the only input is a server-issued quote, the model can't invent a price.
3Check every request
In runTool: is the tool on this session's allow-list? Do the arguments pass the schema (check-out after check-in, dates in the future)? Does the quote belong to this guest?
Why: The model's arguments are untrusted input, exactly like a form posted from the internet. Checks in code hold even when the model is wrong or tricked.
4Gate the money
Before create_booking runs, the runtime needs a recorded "yes" from the guest for this quote: "Hotel Azur, 12–14 June, 312.40 EUR?". Without it, return NEEDS_CONFIRMATION.
Why: Confirmation enforced in code can't be skipped by the model. The guest sees the exact total that will be charged.
5Make retries harmless
Run the booking with an idempotency key built from the session and the quote, and a 10 s timeout. Return short errors, never stack traces.
Why: Timeouts happen and the loop retries. With the key, a second identical request returns the first booking instead of charging twice.
6Send the result back
Reply with a tool_result carrying the same tool_use_id: the booking number, or a short error the model can act on. Then send_email goes to the guest's verified address only.
Why: The model can only report what really happened. The confirmation number now comes from the hotel system, not from the model's imagination.

Tool	Side effect?	Confirmation	Idempotency	Key safeguard
search_hotels	No	No	Safe to retry	Validate dates; cap results to save tokens
get_availability	No	No	Safe to retry	Return only what the model needs
get_quote	No (creates a quote)	No	Same input → same quote	Code computes money, never the model
create_booking	Yes: money	Yes, explicit	Required (key per quote)	Takes quote_id, scoped to the logged-in guest
send_email	Yes: external message	Part of the booking flow	Key per booking	Recipient fixed to the guest's verified email

Classify every tool by its side effects before you write its description.

Interactive simulation

Visual diagram

Guest

Your runtime

LLM

Validate + authorise

search_hotels / get_availability

get_quote

create_booking

send_email

Click a component

The tool-use loop. The model only ever produces requests; the runtime decides and executes.

Analogy

A manager locked in an officeAI tool calling

A manager locked in an officeA stack of request formsAI tool callingTool definitions with inputs
A manager locked in an officeFilling in a form, sliding it under the doorAI tool callingThe model's tool request
A manager locked in an officeThe team outside checks the formAI tool callingYour code validates and authorises
A manager locked in an officePhoning the customer before chargingAI tool callingConfirmation for risky actions
A manager locked in an officeA note slid back: done, or rejectedAI tool callingThe tool result

Where the analogy breaks: a real manager knows when a form looks fishy. The model can be fooled by text it reads, so your team outside must check every form, every time.

Why it matters

Tool calling turns a chatbot into a product. It can check live prices and make real bookings.

It's also where AI causes real damage. The model swaps day and month, and a guest is booked in December instead of June. A request is retried, and the guest is charged twice. A sneaky line hidden in a hotel description tricks the model into emailing private data.

None of these is fixed by a better prompt. Each is fixed by ordinary engineering in your code: checking inputs, checking permissions, asking for confirmation, and (making a repeat do nothing extra). Knowing the loop tells you where each guard goes.

Who is in charge of what

The model

Picks a tool
Fills in the inputs
Can be wrong or tricked

A planner, not a doer.

Your code

Validates every request
Checks permissions
Asks before spending money

The only thing that acts.

Check your understanding

Q1/4Recall

Order one iteration of the tool-calling loop:

1.App executes the real function
2.Model answers or requests another tool
3.App sends a tool_result with the matching id
4.App validates arguments and checks permissions
5.App sends messages + tool definitions to the model
6.Model returns a tool_use block with name and JSON arguments

How confident are you?

Practice ladder

Step 1 · Trace

Trace a booking chat

Scenario Warm-up

The concierge has the five tools from this episode. A guest writes: "Book Hotel Azur in Nice, 12–14 June." Here is the start of the trace:

Model → tool_use: search_hotels({ city: "Nice", check_in: "2026-06-12", check_out: "2026-06-14" })
Runtime → tool_result: 6 hotels, including Hotel Azur
Model → tool_use: get_quote({ hotel_id: "azur", room_type: "double", check_in: "2026-06-12", check_out: "2026-06-14" })
Runtime → tool_result: { quote_id: "Q-881", total: 312.40 }
Model → tool_use: create_booking({ quote_id: "Q-881" })

The guest hasn't confirmed anything yet. Trace what the runtime does at step 5, and what happens next until a booking exists.

Step 2 · Complete

Finish runTool

Code Practice

Lina's runTool has two gaps. Fill in gap A (what to check before a side-effecting tool runs) and gap B (how to call the tool so a retry can't double-charge).

async function runTool(name: string, input: unknown, s: Session) {
  const tool = REGISTRY[name];
  if (!tool || !s.allowedTools.includes(name)) return fail("Unknown or unauthorised tool.");

  const parsed = tool.schema.safeParse(input);
  if (!parsed.success) return fail(`Invalid arguments: ${parsed.error.issues[0].message}`);

  // gap A: ___

  try {
    const data = await withTimeout(/* gap B: ___ */, 10_000);
    return ok(data);
  } catch (e) {
    log.error({ tool: name, e });
    return fail("The booking service is temporarily unavailable. Try again or offer a callback.");
  }
}

Step 3 · Solve

Let the concierge cancel bookings

Design Practice

Maya: "Guests want to cancel through the concierge." Cancelling may cost a fee, depending on the hotel's policy and how close check-in is.

Design the tool(s): names, descriptions, input schemas, what the runtime checks before running, the confirmation flow, idempotency, and the error results the model can receive.

Step 4 · Decide

Auto-rebook when a hotel cancels?

Decision Stretch

Maya: "When a hotel cancels on a guest (overbooking), the concierge should rebook them at a similar hotel automatically, at night, without waiting for the guest." This happens about 30 times a week. The guest's card is on file.

Karim likes it: "Guests wake up already rebooked." Write a short decision: "We chose X over Y because…, accepting the risk of…", and name the checks the runtime needs.

Explain it yourself

Explain how an AI assistant books a hotel room using tools, and where you'd put the safety checks.

0 words