Tool calling

A model calls tools by asking your code to run them

AppModelYour functionquestion + tool definitionstool call: get_order_status("A-1042")the app runs the function{"status": "shipped", "eta": "Thu"}tool result"Your order shipped and arrives Thursday."The model never runs a tool: it asks, your code runs it and returns the result.You describe each tool with a name, a purpose and a JSON schema for its inputs. When a tool would help, the model replies witha structured call instead of text. Your app checks it, runs the real function, and sends the result back; the model then writesthe answer. Every permission, retry and side effect stays in your code.

The model returns a tool call; your app runs it and returns the result.

Tool calling · Going deeper

What a tool definition looks like

{ "name": "get_order_status", "description": "Look up an order by its id. Use when the customer asks where an order is.", "input_schema": { "type": "object", "properties": { "order_id": { "type": "string", "pattern": "^A-[0-9]{4}$" } }, "required": ["order_id"] } }name: what the model writes in its calldescription: when to use it; the modelreads this to decideschema: the inputs, typed and checkedstill validate inputs before runningA tool definition is a name, a description the model reads, and an input schema.The description is a prompt: say what the tool does and when to use it. The schema limits inputs, and your code validates themagain before running anything. Keep tools few and distinct; ten overlapping tools confuse the model more than three clear ones.Bedrock calls these action groups, Foundry and Vertex AI call them tools or functions; the shape is the same.

A name, a description the model reads, and an input schema.

The agent loop

An agent loops: reason, act, observe, until the goal is met

Goal: My flight to Lisbon was cancelled. Get me on one tomorrow morning.ThinkFlight cancelled; find tomorrow-morning optionsActsearch_flights(LHR → LIS, tomorrow, before 12:00)Observe2 flights: 08:10 (3 seats), 11:45 (full)ThinkRebooking rules for a cancellation?Actget_policy("cancellation rebooking")Observefree rebooking on the same route within 48 hActrebook(booking B7Q, flight 08:10)Done"You are on the 08:10 tomorrow, at no charge."ReasonAct: call a toolObserveuntil done, or a step limitAn agent is a model in a loop: reason, call a tool, read the result, repeat until done.Nobody wrote these steps in advance. The model chose each tool from the goal and what it had learned so far, and stopped whenthe goal was met. That flexibility is the point and the risk: the same loop can wander, repeat itself, or take an action youdid not intend. Give it a step limit, a budget, and checks on any action that changes something.

Reason, act with a tool, observe the result, repeat, within a step limit.

The agent loop · Going deeper

Managed agent services

ProviderServiceWhat it addsAWSAmazon Bedrock Agents; Bedrock AgentCoreruntime, memory, gateway for tools, identity, observabilityMicrosoftFoundry Agent Servicetools, knowledge, threads, multi-agent connected agentsGoogleVertex AI Agent Engine; Agent Development Kitmanaged runtime, sessions, memory bankOpen sourceLangGraph, CrewAI, AutoGen, Strandsframeworks you host yourselfAgent services run the loop for you and add memory, tool access, identity and tracing.You still write the instructions, choose the model and define the tools. The service keeps sessions, stores memory, connectstools (often through MCP), signs the agent in to other systems on the user’s behalf, and records every step for debugging.Frameworks give the same building blocks to run on your own infrastructure.

Services run the loop and add memory, tool access and tracing.

Model Context Protocol

MCP lets any AI app use any tool through one standard

Without a standard: 5 × 8 = 40 integrationsapp 1app 2app 3app 4app 5tool 1tool 2tool 3tool 4tool 5tool 6tool 7tool 8With MCP: 5 + 8 = 13 implementationsapp 1app 2app 3app 4app 5tool 1tool 2tool 3tool 4tool 5tool 6tool 7tool 8MCPEach app ships one MCP client; each tool ships one MCP server; any client can use any server.MCP is a standard plug between AI apps and tools, so each side is built once.The Model Context Protocol defines how an AI application discovers a server’s tools, data and prompts and calls them. A toolvendor writes one MCP server, and every MCP-capable assistant, IDE or agent can use it.

MCP turns 5 × 8 = 40 integrations into 5 + 8 = 13.

Model Context Protocol · Going deeper

Hosts, clients, servers and transports

Hostassistant, IDE or agent appMCP client (one per server)MCP clientstdio (local process)Streamable HTTP (remote)MCP server: filestools: read_file, searchresources: file contentsMCP server: ticketingtools: create_ticket, get_ticketprompts: "summarise ticket"Messages are JSON-RPC 2.0. Servers expose tools (actions), resources (data to read) and prompts (templates).Host, client, server: the host app runs one client per server it connects to.Local servers run as a child process over stdio; remote ones use Streamable HTTP with OAuth for sign-in (the older HTTP withserver-sent events transport is deprecated). MCP connects a model to tools; A2A, a separate protocol, connects agents to otheragents. Both are now open projects under the Linux Foundation. Treat a third-party server like any dependency: it can read and act.

Host, client, server; tools, resources, prompts; stdio or Streamable HTTP.

Memory

The context fills with every step; agents summarise and store

151015200k64k128k192kagent steptokens in the contextcontext limit 128kkeep everythingcompact at 96k to a summary6,000 tokens of instructions and tools,+9,000 per step of tool results:the limit is passed at step 14.Long-term memory sits outside:facts and preferences saved to astore and retrieved when relevant.Short-term memory is the context window; it fills, so agents summarise and store.Every tool result is added to the context, so long tasks run out of room, and cost and delay grow with every step before that.Compaction replaces old turns with a summary; tool results can be trimmed to what matters. Long-term memory keeps facts acrosssessions in a database or vector store, retrieved like RAG. AgentCore, Foundry and Vertex AI offer both as managed features.

Short-term memory is the context; compaction summarises it; long-term memory is a store.

Workflows and agents

Fix the steps when you can; use an agent when you must

Workflow: steps fixed in codeExtractCheck policyDraft replysame path every time; easy to test, cheaper, predictableAgent: the model picks the next stepModelsearchdatabaseemailpath decided at run time; flexible, harder to predictPatternWhat it doesRoutingclassify the input, send it to the right prompt or modelParallelrun independent parts at once, or vote on oneOrchestrator and workersone model splits the job and hands parts to othersEvaluator and optimiserone drafts, another critiques, repeatUse a fixed workflow when the steps are known; use an agent only when they are not.Most business processes have known steps and are better as a workflow with model calls inside. Agents earn their cost when thepath depends on what is found along the way. Multi-agent systems give specialists their own tools and a supervisor to route.

Workflow: steps in code. Agent: the model picks the path.

Why long chains fail

Small error rates compound over many steps

0510152025300%25%50%75%100%steps in the taskwhole task succeeds10 steps: 59.9%20 steps: 35.8%99% per step95% per stepSuccess = p per step, multiplied0.95¹⁰ = 0.5990.95²⁰ = 0.3580.99²⁰ = 0.818Assumes each step failsindependently; real errorsalso compound.Small error rates compound: a 95%-reliable step gives a 36% chance of 20 clean steps.Every step is a chance to pick the wrong tool or misread a result. Make each step more reliable (clear tools, validation, retries),make tasks shorter, check progress against the goal along the way, and let the agent stop and ask when it is unsure.

95% per step: 59.9% after 10 steps, 35.8% after 20.

Checks and retries · Going deeper

A check after each step buys back most of the lost reliability

0510152025300%25%50%75%100%steps in the taskwhole task succeeds88.6%59.9%78.5%35.8%checkeduncheckedOne step, with a check after itright first time 95.0%wrong 5.0% caught by the check (80%) 4.0% retry succeeds (95%) 3.8%step reliability now 98.8%20 steps: 35.8% unchecked, 78.5% checked.Price: one check per step, and a retry on about4% of steps.A check that catches most failures, plus one retry, more than doubles the odds of a clean 20-step run.The check need not be a model. The cheapest are in code: did the tool return an error, does the output match its schema, did thetest pass, does the record now exist. A second model as reviewer catches less and costs more. What the check misses stillcompounds, so the other levers stay: fewer steps, and a stop-and-ask rule. Example rates; failures assumed independent.

Verify each step in code where you can, and retry once on a caught failure.

Keeping agents safe

Least privilege and a person’s approval for risky actions

Agent proposesrefund(order A-1042, $480)Allowed?tool in its role, orderbelongs to this userRiskover $100 limit: changesmoneyPerson approvessupport lead clicksapproveExecute and logtool runs with scopedcredentials; every call recordedRead-only calls (look up an order) skip the approval step; actions that move money, delete or send go through it.Also: a step and spending limit per task, a sandbox for code, and an off switch.Give agents the least access that works, and a person’s approval for actions that cannot be undone.An agent acts with whatever permissions its tools hold, so a misread instruction, or a malicious one hidden in a web page oremail (lesson 13), becomes a real action. Scope each tool to the task and the user, require approval above a risk threshold,cap steps and spend, and log every call so it can be traced. OWASP calls the failure of these controls excessive agency.

Least privilege, approval for irreversible actions, limits and logs.

Lesson complete

You finished “Agents and tools”

Nicely done.

Open in a new tab