What makes it an "agent"?

Wrap the model in a loop: ask it for the next action, run that action with real tools, feed the result back, and repeat. The model supplies the reasoning; the loop supplies the hands, the memory, and the brakes.

An Agent Loop (orchestrator) drives everything: it asks the model what to do next, executes that action, observes the result, and loops — until the model says the goal is complete. The model decides; the loop does.

The model picks the next action

The LLM Planner is given the goal, the history so far, and the list of available tools (each with a typed schema). It responds with a structured tool call — a name and JSON arguments — or signals the task is done. Structured output is what makes the next step runnable.

Execute tools — safely

A Tool Executor sits between the model and the world. It validates the requested arguments against the tool’s schema, runs the tool in a sandbox with a timeout, and returns a typed result — success or a structured error. The Tools/APIs are the agent’s hands: search, code, databases, third-party calls.

Observe, then think again

The observation — result or typed error — is appended to the run’s history and fed back to the model, which reasons over it and chooses the next action. Every loop iteration is also written to a Trace Log, so the whole chain of thought→action→result is replayable and debuggable.

Working memory vs long-term memory

Two memories. Working Memory is the scratchpad for the current run — goal, steps taken, observations — trimmed to fit the window. Long-term Memory is a vector store the agent can write durable facts and outcomes to and retrieve in later tasks (RAG, applied to the agent’s own experience).

Budgets, limits and approvals

A Budget & Guard layer wraps the loop: hard caps on steps, tokens and spend, plus approval gates that pause for human sign-off before destructive or costly actions (sending money, deleting data, emailing customers). Bounded autonomy — free to loop, impossible to run away.

Done, or give up gracefully

A run ends two ways: the model emits a "task complete" with the final answer, or the agent hits a wall (blocked, out of budget, repeated failures) and stops cleanly — reporting what it attempted and why it couldn’t finish. Both are returned to the user deliberately. No silent garbage.

Design an AI Agent System — A Guided System Design

System Design · step by stepDesign an AI Agent

Step 1 / 9

Finished this one? 0 / 5 AI System Designs done

More AI System Designs

Design a Conversational AIBuild a production conversational AI system (think ChatGPT) step by step. See how the request path splits an inference gateway from the model servers, how the context window is assembled and token-budgeted, how conversation memory is stored and recalled, how tokens stream back over a persistent connection, and how guardrails gate every prompt and response — through an interactive diagram that grows with each concept.Read →
Design a RAG PipelineBuild a retrieval-augmented generation pipeline step by step. See how documents are chunked and embedded, how a vector store answers semantic search, how two-stage retrieval with reranking finds the best passages, how the prompt is grounded to stop hallucination, and how evals keep a quietly-drifting index honest — through an interactive diagram that grows with each concept.Read →
Design an LLM Inference ServerBuild an LLM inference serving system step by step. See how a request queue absorbs spiky traffic, how the prefill/decode split and continuous batching keep GPUs full, how the KV cache and paged attention make each token cheap, how tensor sharding fits a giant model, and how autoscaling rides demand — all balancing latency against throughput, through an interactive diagram that grows with each concept.Read →
Design a Recommendation SystemBuild a large-scale recommendation system step by step. See how a two-stage retrieve-and-rank funnel picks the best few from millions, how two-tower embeddings and ANN generate candidates fast, how a heavy ranking model scores engagement, how a feature store stays consistent between training and serving, and how the feedback loop keeps recommendations fresh — through an interactive diagram that grows with each concept.Read →

Explore more from Vibe Engines

Handbooks →

Roadmaps →

Prompt Engineering Roadmap

System Designs →