The Prompting Handbook
A short, non-technical field guide built around CRISP (context, role, instruction, specifics and proof by example), with side-by-side bad and good prompts, the eight most common mistakes, and a playground that scores your own prompt. For an engineer the habits carry straight into system prompts: state the output format, say what to avoid, and include one example of a good answer instead of a paragraph describing it.
Attention Is All You Need
Before designing around a model, learn what it does with your prompt: each token attends to the others through queries, keys and values, and position has to be added explicitly because attention on its own ignores order. The breakdown computes attention by hand and in about 30 lines of Python, which is enough to explain why attention cost grows with the square of the context length.
Context Budget & Cost Planner
An interactive planner: set the system prompt, tool definitions, retrieved context, number of turns and price tier, then read the cost per conversation, per day and per month. Use it to answer the cost questions design reviews ask, such as why resent history dominates an agent's input tokens and which lever (trimming retrieval, windowing history, prefix caching or a smaller model) actually moves your number.
Design a RAG Pipeline
The canonical retrieval-augmented generation design, built one component at a time: chunk and embed documents, embed the query, search by similarity, set top-k on the recall and precision dial, rerank, then build a grounded prompt with citations and a way to say it does not know. Several stages end with how they break in production, which is the material for both a RAG design interview and debugging an answer drawn from the wrong passage.
Design a Vector Database
Inside the store that retrieval depends on: exact kNN and the choice of distance metric, IVF with nprobe as the speed and recall dial, HNSW graphs, product quantization to compress vectors, and the recall trap that metadata filters create. It prepares you to pick an index for a given corpus size and latency budget, and to explain how approximate search can silently miss the one document that mattered.
Cosine Similarity
Retrieval ranks chunks by how closely their embeddings point the same way as the query's, and here you write that score yourself against hidden tests. It takes minutes, yet it makes later design choices concrete: why many vector stores normalize embeddings so a plain dot product gives the same ranking, and why mixing metrics between indexing and querying quietly degrades results.
Top-K Retrieval
Given a query embedding and a list of document embeddings, return the indices of the k most similar, best first, with ties broken by the smaller index so results stay deterministic. This brute-force version is exactly what an approximate index tries to match, so it doubles as the ground truth you would measure a vector store's recall against.
RAG Chunking Playground
Paste a real document and compare fixed-size, recursive, sentence and paragraph chunking, with overlap highlighted and a token estimate for every chunk. It turns chunk size from a guess into an experiment: large chunks keep context but dilute relevance and spend tokens at query time, small ones are precise but lose surrounding meaning, and overlap keeps a sentence split across a boundary retrievable.
Design Semantic Search
The full search stack: one embedding model shared by the indexer and the query path, a chunk, embed and index pipeline, hybrid retrieval that adds BM25 for exact terms, a cross-encoder reranker over the shortlist, and model versioning with reindexing. It prepares you for the design question of when vector search alone falls short, and the operational one of what happens to the index when you upgrade the embedding model.
Design an AI Agent System
The move from one call to an agent: a goal and a plan, act and observe loop, typed tool calls, an executor that sandboxes every action, working versus long-term memory, budgets and human approvals, and a graceful way to give up. Work through it before building one, because it answers what reviewers ask first: what stops a runaway loop, what the agent may touch, and how you trace what it did.
Design Multi-Agent Orchestration
An orchestrator over specialist workers, including a section on when not to go multi-agent at all, then shared state versus message passing, a durable workflow engine, typed contracts between agents, parallel fan-out and fan-in, and loop detection. It prepares you to justify the extra moving parts with a concrete win, usually context isolation, and to explain how a long workflow survives a crash halfway through.
Tool-Schema Designer
Compose a function-calling definition field by field (name, description, typed parameters and required flags) and export valid JSON in both the Claude and OpenAI formats. Vague descriptions and loose types are a leading cause of wrong tool choices and invented arguments, so draft the tools your agent design needs here before you write any executor code.
The Agent Evaluations Handbook
Evaluation stops being one accuracy number here: tasks, trials and transcripts, code-based graders, calibrated LLM judges and human review, and metrics for non-determinism that separate succeeding once in k tries from succeeding every time. After the agent designs above, use it to write an eval plan for one of them, including which failures block a release and which only get logged.
Token-Level F1
A small, real metric: lowercase and split a prediction and a reference, count overlapping tokens as a multiset, and combine precision and recall into F1. Building it shows what cheap code-based graders can and cannot catch, which is the judgment you need when deciding where an LLM judge is worth its extra cost and latency.
The Loop Engineering Handbook
Once an agent works, the next step is a system that runs it without you: a trigger, a framed task, the agent's own inner loop, independent verification, and state recorded outside the model. The handbook covers heartbeat, cron, hook and goal loops, hard caps and verifiable stop conditions, cost levers such as model routing and caching, and failure modes like token runaway that compound while nobody is watching.
Self-Improving Agents
A long, hands-on handbook on making an agent better at its own job: repeated sampling and the selection gap, majority vote versus verifiers, building and then breaking a verifier, choosing which traces to keep, and the six places a lesson can land. It prepares you for the hardest question about any improvement loop: how you know the agent got better rather than better at fooling its own grader.
Ship or Stop
Write the release gate for one round of self-improvement: from two rounds of measurements and a canary, return whether to ship and which stop conditions fired, from reward hacking, where the verifier's score rose and the human pass rate did not follow, to a canary read on too few runs. It turns the previous handbook's checklist into code with hidden tests.
Design an LLM Inference Server
What happens after the API call: a router and a queue in front of the GPUs, compute-bound prefill versus memory-bound decode, continuous batching, the KV cache and paged attention, sharding weights across GPUs, and autoscaling on GPU pressure. It prepares you to explain time to first token, why memory rather than compute usually caps how many requests run at once, and what you give up when you trade latency for throughput.
The Senior AI Engineer Interview Handbook
Sixty questions that test whether the previous eighteen steps hold together: multi-tenancy on a shared platform, model routing, rolling back a bad prompt, a provider outage, hallucinations in production, agent state and runaway cost, retrieval at scale, and evals. Use it as the final self-test, answering each question aloud and returning to the step that covers any answer you cannot give.