A written version of the interactive roadmap above — every station, what you'll learn, and a small thing to build — laid out for reading, reference and search.
Model I/O Start here
F1. The Messages API
Beginner · 45 min
A harness talks to a model through one primitive: a list of messages with roles (system, user, assistant, tool) exchanged over request and response. Learn the shape of a completion call, how turns accumulate, and why everything the model knows this turn is exactly what you put in the array. This is the socket the whole harness is built on.
Skills: Roles & message turns · Request / response shape · Stateless calls · The assistant turn
Build it: Trace one agent turn as a raw messages array. What is in it, and what does the model return — text, or a tool call?
F2. Tokens, Context & Streaming
Beginner · 55 min
The model reads and writes tokens, not characters, inside a fixed context window — and streams them back one at a time. Learn how text becomes tokens, why the window is a hard budget every message competes for, and how streaming lets a harness react before generation even finishes.
Skills: Tokenization · The context-window budget · Streaming responses · Counting tokens
Build it: Estimate the tokens in a system prompt plus a 10-turn history. When does it threaten the window — and what gets dropped first?
F3. Structured Output
Intermediate · 55 min
Free-form text is unusable as an API. Force the model to emit valid JSON against a schema — through JSON mode, tool calls, or constrained decoding — so your code can parse it every time. Learn to design output schemas and to recover gracefully when the model still drifts.
Skills: JSON mode · Output schemas · Constrained decoding · Parse & repair
Build it: A model must return {intent, args}. Design the schema and one guard for when it returns prose instead of JSON.
F4. System Prompt & Steering
Intermediate · 60 min
The system prompt is the harness's constitution — the standing instructions, tools, format rules and persona the model obeys every turn. Learn to write one that is specific, ordered, and stable enough to cache, and why most agent misbehaviour is a prompt bug, not a model failure.
Skills: System vs user turns · Instruction design · Guardrails in-prompt · Stable, cacheable prefixes
Build it: Write a system prompt for a support agent: what it can do, its output format, and one hard rule it must never break.
F5. Sampling & Determinism
Intermediate · 50 min
The same prompt can give different answers. Learn temperature, top-p and top-k — the dials that shape the model's randomness — and how seeds and low temperature buy the reproducibility a harness needs for tools and tests, without going brittle.
Skills: Temperature, top-p, top-k · Greedy vs sampled · Seeds & reproducibility · Determinism for tools
Build it: Your agent must call a tool the same way every time for a given input. What sampling settings get you there — and what do you lose?
F6. Prompt Caching
Intermediate · 45 min
Every turn re-sends the same long prefix — system prompt, tools, history. Prompt-prefix caching lets the provider skip re-processing that stable head, cutting latency and cost dramatically. Learn what makes a prefix cacheable and why cache-friendliness shapes how you order a prompt.
Skills: Prefix caching · Cache-friendly ordering · Latency & cost wins · Cache invalidation
Build it: Reorder a prompt so the stable parts sit first and the volatile parts last. Why does that one change slash the bill?
The Agent Loop The craft
T1. Tool / Function Calling
Intermediate · 60 min
Tool calling is what turns a text model into an agent: you describe functions, the model decides when to call one and emits structured arguments, your code runs it and feeds the result back. Learn the full round-trip — and why the model never runs anything itself, the harness does.
Skills: Tool definitions · Model-emitted calls · Executing & returning results · Multi-tool turns
Build it: Give a model a get_weather tool. Walk the round-trip: what the model emits, what your code does, what goes back into the messages.
T2. The Agent Loop
Advanced · 65 min
Plan, act, observe, repeat. The agent loop feeds the model its own tool results and lets it decide the next step until the task is done. Learn loop control — stopping conditions, step limits, and detecting when the model is stuck or looping — the beating heart of every harness.
Skills: Plan / act / observe · Stopping conditions · Step limits · Loop & stuck detection
Build it: Design the stop condition for a research agent. How do you tell "done" from "gave up" from "looping forever"?
T3. Tool Schema Design
Advanced · 55 min
The tool schema is the contract the model codes against; a vague one is the top cause of tool-calling failures. Learn to write names, descriptions, typed parameters and required flags that make the model pick the right tool and fill it correctly — and to keep the toolset small and distinct.
Skills: Names & descriptions · Typed params & enums · Required vs optional · Minimal, distinct toolsets
Build it: A model keeps calling the wrong tool between two similar ones. Rewrite their schemas so the choice is unambiguous.
T4. Context Management
Advanced · 65 min
The window fills as the loop runs — tool outputs, history, retrieved docs — and once it overflows the agent forgets. Learn compaction: summarizing old turns, trimming tool noise, and deciding what to keep so the agent stays coherent over long tasks without blowing the budget.
Skills: Compaction & summarization · Trimming tool output · What to keep vs drop · Windowing history
Build it: A 40-step agent overflows its window at step 25. Design a compaction rule that keeps it coherent all the way to step 40.
T5. Memory
Advanced · 60 min
Context is short-term memory that dies with the session; a harness needs more. Learn long-term memory — writing facts to a store and retrieving the relevant ones back into context — and the split between scratchpad, episodic, and semantic memory that lets an agent learn across runs.
Skills: Short vs long-term memory · Write & retrieve · Scratchpad / episodic / semantic · Memory into context
Build it: An agent should remember a user's preferences across sessions. What do you store, and how does it come back into the next prompt?
T6. Sandboxed Execution
Advanced · 60 min
The moment an agent runs code or commands, safety becomes the harness's problem. Learn to sandbox execution — isolation, resource limits, network and filesystem boundaries — so a model's mistake, or an injection, can't escape. The harness, not the model, owns the blast radius.
Skills: Isolation & containers · Resource limits · Network / FS boundaries · Containing failure
Build it: Your agent executes model-written Python. List three things the sandbox must prevent — and how each is enforced.
Production Operate & scale
P1. Permissions & Approvals
Advanced · 55 min
Not every action should run unattended. Learn to gate dangerous tools behind human approval, scope what an agent may touch, and design the permission model — allow, ask, deny — that lets a harness act autonomously where it is safe and pause where it is not.
Skills: Allow / ask / deny · Human-in-the-loop · Action scoping · Approval UX
Build it: An agent can read files freely but must ask before deleting. Design the permission check that enforces that split.
P2. Subagents & Orchestration
Advanced · 60 min
One agent, one context, hits limits. Learn to spawn subagents — each with its own window and tools — and orchestrate them: delegating a subtask, collecting the result, and keeping the parent's context clean. The pattern behind planners, workers, and multi-agent systems.
Skills: Subagents & delegation · Isolated sub-contexts · Result aggregation · Planner / worker patterns
Build it: A big task blows one context window. Split it across subagents — what does each get, and how does the parent recombine them?
P3. Retries, Timeouts & Recovery
Advanced · 55 min
Models rate-limit, tools time out, JSON comes back malformed. A production harness expects failure: learn retries with backoff, timeouts and fallbacks, and how to feed an error back to the model as an observation it can recover from — instead of crashing the loop.
Skills: Retries & backoff · Timeouts & fallbacks · Error-as-observation · Graceful degradation
Build it: A tool returns a 500 mid-loop. Show two recovery paths: one your code handles, one the model handles.
P4. Evals in the Loop
Advanced · 60 min
You cannot ship what you cannot measure. Learn to evaluate a harness — offline test suites, LLM-as-judge scoring, and in-loop self-checks where the agent verifies its own work before finishing. Evals are the regression test that keeps a prompt change from silently breaking everything.
Skills: Offline eval suites · LLM-as-judge · In-loop self-check · Regression on prompt changes
Build it: You tweak the system prompt. What eval tells you — before users do — whether the agent got better or worse?
P5. Observability & Tracing
Advanced · 55 min
When an agent does something weird, you need the tape. Learn to trace every turn — prompts, tool calls, results, tokens, latency, cost — so you can replay a run, find where it went wrong, and debug behaviour that only shows up in production.
Skills: Per-turn tracing · Prompt & tool logging · Token / latency / cost metrics · Replay & debugging
Build it: An agent gave a bad answer once, in production. What must your trace have captured for you to reconstruct why?
P6. Cost & Latency Control
Advanced · 55 min
A harness that is correct but slow or expensive will not ship. Learn the levers — model routing (a cheap model for easy turns), caching, trimming context, and parallelizing tool calls — that keep an agent fast and affordable at scale without dropping quality where it counts.
Skills: Model routing · Caching & context trimming · Parallel tool calls · Quality vs cost tradeoffs
Build it: Your agent averages 12 model calls per task at frontier prices. Name three changes that cut cost without hurting results.