Episode 117 · Booked for December

Designing tool schemas the model can't misuse

Monday, 9:30: in nine days, 23 guests who asked to move their stay to "12/06" were moved to 6 December instead of 12 June. Every one of them confirmed, and the cause is a single free-text field in a tool's schema.

The model only sees your tool's name, description and inputs. Design them well and it picks the right tool. Design them badly and it guesses.

30 min advancedTool names and descriptions as the model's only docsParameter design: enums, required fields, server-issued idsTool granularity: few coarse tools vs many fine onesSeparating read tools from write toolsStrict tool schemas and constrained argumentsTool schemas & descriptions

Problem
A hotel assistant has 30 tools. It calls the wrong one, invents arguments, and sometimes cancels when it should only look up.
Why the simple system fails
Tools designed like internal functions (vague names, free-text parameters, one god-tool, read and write mixed) give the model ambiguous choices and give your code nothing to check.
Engineering concept
Treat the tool set as an API for a reader who only sees the docs: sharp names and descriptions, typed and constrained parameters, the right granularity, and read/write separation.
Technology
JSON Schema with enums, required fields and additionalProperties: false; strict/constrained modes; Zod for runtime validation; tool subsets per task.
Trade-offs
Fewer coarse tools are safer and cheaper in tokens but less flexible; strict schemas guarantee shape, not correctness; every tool you add costs tokens and selection accuracy.

The problem

2 min

Monday, 9:30. Nine days after the concierge got 31 new tools.

Omar · On-call SRE
23 stays moved to December in nine days. All through update_booking, all confirmed by the guest.
Maya · Product manager
They typed "move me to 12/06": 12 June. The confirmation showed "12/06" back, so they said yes.
Lina · Senior engineer, your mentor
You first: what would you change?

The big idea

4 min

One loose field, 23 wrong stays

The big idea

Design tools like an API for a reader who only sees the docs, and leave no room to guess.

"Always use ISO dates" in the prompt makes the bug rarer. A typed field plus a check in code makes it impossible. The model sees no codebase and can't ask questions: the schema is its only manual.

How it works

5 min

Designing a tool for a reader who only sees the docs

1
Name the intent
One tool per thing a guest can say in a sentence: change_reservation_dates, not update_booking.
2
Describe when
The says what it does, when to use it, when NOT (and what to use instead), and what it returns.
3
Type every field
Dates with format: date, an enum (a fixed list, like room codes DBL, DBL-SV, STE) for closed sets, ranges for numbers, no extra properties.
4
Ids, not values
Prices and dates live in a server-issued quote_id. Identity comes from the session. Anything deciding whose data, how much money or what's allowed never travels in arguments.
5
One risk level, checked in code
A write tool (it changes something) never shares a tool with a read, so it can be gated. A guarantees shape, not truth: the handler still checks ownership and dates.
6
Send a small set
Each chat type sees the 5–6 tools it needs: ~900 tokens of definitions instead of 4,650, and fewer wrong picks.

Watch a decision being made

4 min

Worked example

Lina and you redesign the date change

Replace update_booking(booking_id, fields: string) with tools the model can't misuse for "Move my stay at Hotel Azur to 12–14 June".

1Split by intent and risk
"What do guests actually ask?" get_reservation (read), quote_date_change (read, creates a quote), change_reservation_dates (write), cancel_reservation (write).
Why: The runtime gates by tool. A read and a write in one tool can't be gated separately.

2Type the dates

In quote_date_change: required check_in and check_out with format: date.

json

"check_in":  { "type": "string", "format": "date", "description": "YYYY-MM-DD" },
"check_out": { "type": "string", "format": "date", "description": "After check_in, max 30 nights" }

Why: Required fields make the model ask instead of guessing; formats turn creative answers into rejected ones.

3Pass ids, check truth
The write takes only reservation_id and quote_id. The handler checks the reservation belongs to ctx.user, the quote to the reservation, check-in in the future.
Why: Strict mode guarantees shape; only your code knows whose reservation it is and what today is.
4Offer a small set
"Modify my stay" chats get 5 tools: ~750 tokens instead of 4,650.
Why: Fewer, distinct choices mean fewer wrong picks, and every call gets cheaper.

In real life

3 min

Tool design is API design

Stripe's REST API: narrow and explicit

Widely known for resource-by-resource endpoints (create a refund, capture a payment) and detailed docs
Each operation has its own parameters and rules
Easy to put a check or permission on each

Like intent tools: one action, one risk level, easy to gate.

GitHub's GraphQL API: one flexible query

Offered publicly next to its REST API
The caller composes exactly the data it wants
Great for reading; harder to gate per action

Like a flexible read tool.

What goes wrong

3 min

Exposing your REST API 1:1 as tools

Endpoints overlap, use jargon and mix risk levels.

Instead

Tools around guest intents, one risk level each, built on top of the API.

Trusting strict mode as validation

It guarantees shape, not valid business values or permissions.

Instead

Keep validation and authorisation in the handler.

Descriptions that say only what, not when

"Updates a reservation" doesn't help choose between three similar tools.

Instead

Say when, when not (and what instead), prerequisites and what it returns.

Trade-offs

3 min

Tool shape	Choose when	You pay
Few intent-level tools	Guest-facing; money, reservations, personal data	Each new action needs a hand-made tool
Many API-level tools	Internal agents composing new workflows	Wrong-tool calls, more tokens, rules across calls
One flexible read tool	Harmless exploration	Hard to gate per action

Thursday, 14:00. Planning the next sprint.

Karim · CTO
Nine hand-made tools took a week. Loyalty and spa are coming. Just expose the endpoints.
Lina · Senior engineer, your mentor
Exposing them is what moved 23 stays to December. Each guest-facing write needs its own intent tool.
Karim · CTO
Then every new feature waits for someone to hand-write a tool.

Your job as tech lead

4 min

Claude Code can write the code for this. You still own the decisions, and you have to know what good looks like to accept or reject its work.

Decisions you own

The tool set: intent tools (9, with 5–6 per chat type) instead of 31 PMS endpoints exposed 1:1.
Which values the model may fill, and which come from code: identity from the session, prices from a quote id.
Exact formats for risky inputs: ISO dates, enums for room types and reasons.

Tell Claude Code

We exposed the PMS as 31 tools (~4,650 tokens per call); the agent picks the wrong tool in ~1 in 10 multi-step chats, and `update_booking(booking_id, fields: string)` moved 23 stays to December. Replace it with intent tools in one registry: clear names, descriptions saying when and when not, typed parameters (ISO dates, enums), a read/write flag, the guest from the session. Validate in the handler with the same schema. Show me the tool list first, then replay 200 recorded chats and report wrong-tool rate and tokens per call.

Check before you accept

Read the tool list: could two tools be confused? Does each description say when to use it?
No parameter controls identity, an amount or a recipient.
Dates are ISO in the schema, and the confirmation spells them out ("Fri 12 June 2026").

Red flags

A god tool like update_booking(fields: string) survives.
Validation dropped from the handler because strict mode is on.
Wrong-tool choices "fixed" with a prompt line instead of merging or renaming tools.

Check yourself

2 min

Q1/3Design

You need a refund tool for a hotel assistant. Which definition is safest?

How confident are you?

Next episode: Friday evening, the hotel system slows down, and every guest question can hit it up to 27 times. This time you lead.

Finished?

Mark it complete to update your skill map.

Next: Tool errors, retries and loops that end

Go deeper (optional)You don't need this to understand the lesson. Open it when a project needs the details: Technical deep dive, Interactive simulation, Visual diagram, Analogy, Why it matters, Check your understanding, Practice ladder, Explain it yourself.

Technical deep dive

From endpoint to intent

Worked example

Lina and you redesign the date change

Replace update_booking(booking_id, fields: string) with tools the model can't misuse for "Move my stay at Hotel Azur to 12–14 June".

1Split by intent and risk
List what guests actually ask: look up a stay, price a date change, apply it, cancel. That's get_reservation (read), quote_date_change (read, creates a quote), change_reservation_dates (write), cancel_reservation (write).
Why: The runtime gates by tool. If a read and a write share one tool, you can't confirm one without nagging on the other.
2Write the description for a stranger
"Move an EXISTING reservation to new dates at the same hotel. Only after quote_date_change and the guest's yes. Does NOT change room or hotel (use rebook_reservation). Returns the updated reservation and the price difference."
Why: The model chooses between tools by reading descriptions. "Updates a booking" says what; it never says when, or which sibling to use instead.
3Type every field
In quote_date_change: check_in and check_out with format: date, reservation_id with a pattern, additionalProperties: false, all required.
json
```
"check_in":  { "type": "string", "format": "date", "description": "YYYY-MM-DD" },
"check_out": { "type": "string", "format": "date", "description": "After check_in, max 30 nights" },
"reservation_id": { "type": "string", "pattern": "^[A-Z]{3}-\\d{5}$" }
```
Why: Required fields make the model ask the guest instead of guessing. Formats and patterns turn creative answers into rejected ones.
4Pass ids, not values
The write tool takes only reservation_id and quote_id. The quote stores the dates and the price; the guest's identity comes from the session.
Why: The model can't invent a price or act as another guest, and the guest confirms exactly what the quote contains.
5Check the truth in code
The handler checks: the reservation belongs to ctx.user, the quote belongs to that reservation and hasn't expired, check-in is in the future.
Why: Strict mode guarantees the shape. Only our code knows whose reservation it is and what today's date is.
6Offer a small set
For "modify my stay" chats, send only get_reservation, quote_date_change, change_reservation_dates, cancel_reservation, search_availability: 5 × 150 ≈ 750 tokens instead of 4,650.
Why: Fewer, distinct choices mean fewer wrong picks, and every call of the loop gets cheaper.

Design	Example	Problem	Better
God tool	manage_booking(action, fields)	One schema for everything; can't gate cancel differently from lookup	Split by intent: get_ / change_dates / cancel_
API mirror	patch_reservation, put_reservation	Mirrors HTTP, not intent; overlapping names	Name tools after what the guest asks for
Too fine	reserve_room → set_price → charge_card	The model orchestrates a money flow; partial failures	One create_booking(quote_id); code runs the steps
Near-duplicates	find_hotels, search_hotels, lookup_hotels	The model picks randomly between them	One tool with optional filters

Granularity is about who orchestrates: the model for harmless exploration, your code for money and anything irreversible.

Interactive simulation

Visual diagram

Guest

Agent runtime

Tool registry

LLM

Schema + policy check

Read tools

Write tools

Click a component

Where a tool definition is used: the model reads it to choose, your runtime uses the same schema to check.

Analogy

A temp receptionist with paper formsAn AI model with tools

A temp receptionist with paper formsA form titled "Booking stuff"An AI model with toolsA vague tool name
A temp receptionist with paper formsTick boxes: single, double, suiteAn AI model with toolsA fixed list of allowed values
A temp receptionist with paper formsCancel form in a different colourAn AI model with toolsWrite tools kept apart and guarded
A temp receptionist with paper formsBack office calls the guest firstAn AI model with toolsConfirmation before risky changes
A temp receptionist with paper forms60 forms in one drawerAn AI model with toolsToo many overlapping tools

Where the analogy breaks: a temp learns your habits over weeks. The model starts fresh every call and knows only what's written on the form.

Why it matters

Tool design is the cheapest way to make an agent better.

Most "the agent did something dumb" bugs come from overlapping tools or fuzzy inputs, not from a weak model. Renaming two tools often helps more than buying a bigger model.

It's also about safety. A tool called refund(amount) can't tell "refund the deposit" from "refund 10,000 euros". A tool called refund_deposit(reservation_id) can.

And it's about cost. Every tool definition is sent with every model call. Twenty wordy tools add up fast across a long conversation.

Check your understanding

Q1/3Reasoning

Match each design choice to the failure it prevents:

Enum for room_type

quote_id instead of price

user from session, not arguments

Separate read and write tools

How confident are you?

Practice ladder

Step 1 · Trace

Trace a room-type request

Scenario Warm-up

The concierge has this read tool:

{
  "name": "search_availability",
  "input_schema": {
    "type": "object",
    "properties": {
      "hotel_id":  { "type": "string", "enum": ["LIS-01", "AMS-01", "NCE-02"] },
      "check_in":  { "type": "string", "format": "date" },
      "check_out": { "type": "string", "format": "date" },
      "guests":    { "type": "integer", "minimum": 1, "maximum": 6 },
      "room_type": { "type": "string", "enum": ["SGL", "DBL", "DBL-SV", "STE"] }
    },
    "required": ["hotel_id", "check_in", "check_out", "guests"],
    "additionalProperties": false
  }
}

A guest at Hotel Azur in Nice (NCE-02) writes: "Any double with sea view for me and my partner, 14 to 12 June?" Strict mode is on. Trace what the model can send, what the schema guarantees, and what our handler must still catch.

Step 2 · Complete

Finish the cancel tool

Design Practice

Lina started the definition of cancel_reservation. Fill in the description and the input schema (fields, types, required, extra properties).

defineTool({
  name: "cancel_reservation",
  kind: "write",
  description: ___,
  input: ___,
  handler: async (args, ctx) => pms.cancel(ctx.user, args),
});

Step 3 · Solve

Redesign the spa god tool

Design Practice

The spa team built one tool: spa(action: string, details: string), where action is "search", "book", "cancel" or "price", and details is free text like "massage tomorrow 3pm for 2, 60 min". Redesign it as a tool set. For each tool give the name, read or write, a one-sentence description including when NOT to use it, and the parameters with types, enums and which are required.

Step 4 · Decide

40 tools and growing

Decision Stretch

By next quarter the concierge's registry will hold about 40 tools (bookings, loyalty, spa, restaurants). Two proposals:

A: send all 40 on every call; models are good now.
B: a small classifier picks the chat type first ("modify stay", "spa", "loyalty"…) and sends only that type's 5–8 tools.

Each definition is ~150 tokens and a typical chat makes 3 model calls. Write a short decision: "We chose X over Y because…, accepting the risk of…", with at least one number.

Explain it yourself

Explain to a teammate why exposing your booking REST API 1:1 as tools is a bad idea, and what you'd do instead.

0 words