The big idea
What makes it an "agent"?
A chatbot answers a question in one shot. An agent is handed a goal — "find the cheapest flight and book it" — and has to figure out the steps, take actions in the real world, react to what happens, and keep going until it’s done. How do you build something that acts, not just answers?
Wrap the model in a loop: ask it for the next action, run that action with real tools, feed the result back, and repeat. The model supplies the reasoning; the loop supplies the hands, the memory, and the brakes.
How to read this: Each step opens with a real design decision — you make the call before I show you what ships. Watch the loop grow, hover any box, replay the flow. At the end break a tool to see why error handling is the whole game. Hit Begin.
Step 1 · The skeleton
A goal and a loop
The user gives a goal, not a question. A single model call can’t book a flight — it can only produce text. What structure turns "produce text" into "get something done"?
Design decision: The user states a multi-step goal. What runs it?
The call: A loop that asks the model for the next action, runs it, and feeds back the result. — The agent loop alternates thinking and doing: decide → act → observe → repeat, until the goal is met. This is the spine of every agent.
An Agent Loop (orchestrator) drives everything: it asks the model what to do next, executes that action, observes the result, and loops — until the model says the goal is complete. The model decides; the loop does.
Plan → act → observe: The loop alternates reasoning (model picks an action) and acting (the system runs it and reports back). Looping over that is what lets an agent handle tasks no single call could.
Step 2 · Let it decide
The model picks the next action
Inside the loop, something has to choose what to do next — search? query a DB? finish? That judgment is exactly what the LLM is good at. But how does free-form text become a concrete, runnable action?
Design decision: How does the agent turn the model’s reasoning into an actual action?
The call: Have the model emit a structured tool call (name + typed arguments). — The model outputs a function call — tool name and JSON arguments matching a schema — which the loop can validate and execute deterministically.
The LLM Planner is given the goal, the history so far, and the list of available tools (each with a typed schema). It responds with a structured tool call — a name and JSON arguments — or signals the task is done. Structured output is what makes the next step runnable.
Tool calling: Tools are described to the model as typed functions. The model’s job becomes "which function, with what arguments?" — a constrained, checkable decision instead of free-form prose.
Step 3 · Give it hands
Execute tools — safely
The model asked to run a tool. But its arguments might be malformed, the tool might be dangerous (delete data, spend money), and it might hang. You can’t just blindly run whatever it emits. How do you execute safely?
Design decision: The model requests a tool call. How do you run it?
The call: Validate against a schema, run sandboxed with a timeout, return a typed result. — A Tool Executor checks the arguments, runs the tool in a sandbox with limits, and returns a structured result OR a structured error — both readable by the model.
A Tool Executor sits between the model and the world. It validates the requested arguments against the tool’s schema, runs the tool in a sandbox with a timeout, and returns a typed result — success or a structured error. The Tools/APIs are the agent’s hands: search, code, databases, third-party calls.
The executor is the airlock: Every action passes through one guarded boundary: validate → sandbox → timeout → typed result. Centralizing execution is what lets you add limits, logging and safety in one place.
Step 4 · Close the loop
Observe, then think again
The tool ran and returned something — a result, or an error. The model doesn’t know what happened yet. How does the outcome get back into its reasoning so it can decide the next move?
Design decision: A tool returned a result (or failed). What happens next?
The call: Feed the observation back into the model and loop. — The result (or typed error) is appended to the agent’s history and handed back to the model, which reasons about it and picks the next action. That’s the "observe" in plan-act-observe.
The observation — result or typed error — is appended to the run’s history and fed back to the model, which reasons over it and chooses the next action. Every loop iteration is also written to a Trace Log, so the whole chain of thought→action→result is replayable and debuggable.
Observability is non-negotiable: Agents are non-deterministic and multi-step, so when one misbehaves you need the full trace — every thought, call and observation — to understand why. The event log isn’t optional; it’s how you debug, eval and bill.
Step 5 · Make it remember
Working memory vs long-term memory
Within one task the agent must remember what it has already tried. Across tasks, it’s wasteful to relearn the same facts every time. These are two different memory needs. How do you serve both?
Design decision: The agent needs to remember within a task AND across tasks. How?
The call: Working memory for the current run + vector long-term memory across runs. — A working-memory scratchpad holds the live task; a vector-backed long-term store lets the agent recall facts and past outcomes in future tasks.
Two memories. Working Memory is the scratchpad for the current run — goal, steps taken, observations — trimmed to fit the window. Long-term Memory is a vector store the agent can write durable facts and outcomes to and retrieve in later tasks (RAG, applied to the agent’s own experience).
Two scopes of memory: Working memory answers "what have I done in this task?" Long-term memory answers "what have I learned across tasks?" Conflating them either overflows the prompt or forgets everything.
Step 6 · Don’t let it run away
Budgets, limits and approvals
A looping model can spin forever, rack up a huge bill, or take a destructive action. Autonomy without limits is a liability. How do you keep an agent on a leash without making it useless?
Design decision: What stops an agent from looping forever or doing something destructive?
The call: A budget/guard layer: step + token + spend caps, plus approval gates on risky actions. — Hard limits cap steps, tokens and cost; an approval gate pauses for human sign-off before destructive or expensive actions. Independent of the model’s judgment.
A Budget & Guard layer wraps the loop: hard caps on steps, tokens and spend, plus approval gates that pause for human sign-off before destructive or costly actions (sending money, deleting data, emailing customers). Bounded autonomy — free to loop, impossible to run away.
Bounded autonomy: The art of agents is giving freedom inside limits. Step/cost budgets stop runaway loops; approval gates put a human in front of irreversible actions. The model reasons; the guard decides what it’s allowed to do.
Step 7 · Finish cleanly
Done, or give up gracefully
Eventually the agent either achieves the goal or can’t. Both endings need to be deliberate — not "loop exhausted, returns garbage." How does a run actually end?
Design decision: How should an agent run terminate?
The call: On an explicit "done" from the model, or a clean stop when blocked or out of budget. — The model signals completion with the final answer; if it’s blocked or hits a limit, the loop stops and reports what it tried and why it couldn’t finish.
A run ends two ways: the model emits a "task complete" with the final answer, or the agent hits a wall (blocked, out of budget, repeated failures) and stops cleanly — reporting what it attempted and why it couldn’t finish. Both are returned to the user deliberately. No silent garbage.
Graceful failure: A trustworthy agent knows when to quit. "I tried X and Y, hit this blocker, here’s where I got" beats an infinite loop or a confident lie every time.
The payoff
You built an AI agent
From a one-shot chatbot to an autonomous agent: a plan-act-observe loop, structured tool calls, a guarded executor, two scopes of memory, full tracing, hard budgets, approval gates, and graceful endings.
Now break a tool and watch the agent hit an error mid-loop — and see why the executor’s typed errors, bounded retries and step budget are the difference between recovery and a runaway death spiral.
- Agent Loop — plan → act → observe → repeat until done
- Tool calling — model emits typed, schema-checked function calls
- Tool Executor — the airlock: validate, sandbox, timeout, typed result
- Observe + Trace — feed results back; log every step for debug/eval
- Two memories — working (this run) + long-term (across runs)
- Budget & Guard — step/token/$ caps + approval on risky actions
- Graceful end — "done", or stop cleanly and report when blocked
- Error handling — typed errors + bounded retries beat death spirals