Every AI engineer builds on Python and a working feel for machine learning. Arrays and tensors, training vs inference, features and loss, overfitting — you do not need to invent models, but you need the vocabulary to reason about the ones you use.
Build itLoad a small dataset, train a simple classifier with scikit-learn, and read its accuracy. Then break it — shuffle the labels — and watch accuracy collapse to chance.
✓ CheckpointExplain what a model actually learned when accuracy collapses to chance after you shuffle the labels — and why that is the most informative experiment on this station.
An LLM predicts the next token, over and over. Under the hood is the Transformer — self-attention lets every token look at every other token at once. Understanding tokens, attention, and sampling (temperature, top-p) demystifies everything the model does downstream.
Build itTokenize a paragraph with a real tokenizer and count the tokens. Then generate the same prompt at temperature 0 and 1.0 five times each, and compare how much the output varies.
✓ CheckpointExplain why the same prompt at temperature 0 can still be wrong in the same way every time, and what that tells you about where the error lives.
Turn text into a vector and meaning becomes geometry: similar things sit close together. Embeddings and cosine similarity are the engine under semantic search, recommendations, and retrieval — the single most useful representation in applied AI.
Embeddings
Cosine similarity
Nearest-neighbour search
Semantic vs keyword match
Build itEmbed 20 short sentences, then for a query sentence rank the rest by cosine similarity. Check whether the top matches actually mean the same thing.
✓ CheckpointExplain why two sentences that share no words can sit close together in embedding space, and name a case where that closeness misleads you.
The prompt is your primary interface to the model. System vs user roles, clear instructions, few-shot examples, and asking the model to reason before answering are the highest-leverage, lowest-effort levers you have on quality.
System / user roles
Few-shot examples
Instruction clarity
Reason-then-answer
Build itTake one task and write it three ways: zero-shot, few-shot, and reason-then-answer. Run each on 10 inputs and tally which is most reliable.
✓ CheckpointTake a prompt that failed and say whether the fix belongs in the instruction, the examples, or the context you supplied — “write a better prompt” is not an answer.
Every token costs money and latency, and every model has a finite context window. Knowing how text becomes tokens, what fits in the window, and how pricing works is the difference between a demo and something you can afford to run.
Tokenization
Context-window budgeting
Pricing math
Model size vs cost
Build itEstimate the token count and dollar cost of a long RAG prompt before sending it. Then trim it 30% and re-estimate — how much did you save?
✓ CheckpointCompute the cost of one request from its token count and price per million, then say which part of the prompt you would cut first and what it would break.
To wire an LLM into software you need reliable structure, not prose. JSON mode, output schemas, and function/tool calling let the model return data your code can trust — and let it call your functions with well-typed arguments.
JSON mode
Output schemas
Function / tool calling
Validation & retries
Build itDesign a tool schema for a "get_weather(city, unit)" function, then prompt the model to call it correctly on 10 varied requests. Count schema violations.
✓ CheckpointExplain why asking a model for JSON is not the same as guaranteeing JSON, and what your code does on the day it returns prose instead.
6 rungs · ~6h · ends at Multi-Agent Orchestration.
0/6 done
1
Retrieval-Augmented Generation
T1 · Intermediate · 60 min
RAG lets a model answer from your documents instead of only its frozen memory: retrieve the most relevant chunks, then generate grounded in them. It is the backbone of every "chat with your docs" product and the main tool for cutting hallucination.
Chunking
Retrieve-then-generate
Grounding & citations
Hallucination reduction
Build itBuild a minimal RAG loop: embed a document, retrieve top-k chunks for a question, and prompt the model to answer using only those chunks, with citations.
✓ CheckpointExplain why a retrieval system can return the correct document and still produce a wrong answer, and which half of the pipeline you would investigate first.
At scale you need fast nearest-neighbour search over millions of vectors. Vector databases use approximate indexes (HNSW, IVF) to trade a little accuracy for huge speed, and pair with chunking strategies and re-rankers to make retrieval actually good.
ANN indexes (HNSW / IVF)
Chunking strategy
Hybrid search
Re-ranking
Build itIndex a few thousand chunks in a vector DB and compare exact vs approximate search: measure recall and latency at different index settings.
✓ CheckpointExplain what approximate means in approximate nearest neighbour, and what you traded to get the speed.
An agent thinks, acts, observes, and repeats — the ReAct loop. Give a model tools plus a reasoning loop and it can pursue multi-step goals: search, call APIs, read results, and adjust. Most of the hard work is tool schemas, memory, and loop control.
ReAct loop
Tool schemas
Short/long-term memory
Loop termination & recovery
Build itBuild a 2-tool agent (search + calculate). Run it on 5 multi-step questions and log where it loops, hallucinates a tool call, or stops too early.
✓ CheckpointExplain why an agent that never stops is a design failure rather than a bug, and name the stopping conditions yours actually has.
Ask a model to show its working and multi-step accuracy jumps. Chain-of-thought, self-consistency (sample many chains, vote), and modern reasoning models all rest on one idea: give the model room to think before it commits to an answer.
Chain-of-thought
Self-consistency
When reasoning helps
Reasoning models
Build itTake 10 word problems. Answer each directly, then with "let's think step by step". Measure the accuracy gap.
✓ CheckpointExplain what chain-of-thought buys you and what it costs per request — and name a task where it makes the answer worse.
When prompting and RAG are not enough, fine-tune. LoRA freezes the base model and trains tiny adapters — cheap, fast, and swappable — while preference tuning (DPO) aligns behaviour. The key skill is knowing when fine-tuning beats retrieval, and when it does not.
RAG vs fine-tuning
LoRA / QLoRA adapters
Preference tuning (DPO)
Dataset curation
Build itSketch a decision: for a given task, argue whether RAG, fine-tuning, or both is right — and what data you would need for the fine-tune.
✓ CheckpointSay whether your problem is the model not KNOWING something or not BEHAVING a certain way, and defend the choice that follows.
Some problems are better split across specialised agents — a planner that decomposes work, workers that execute, and handoffs between them. Orchestration is about coordination: routing, shared state, and stopping the whole system from looping or drifting.
Planner / worker patterns
Agent handoffs
Shared state
Failure isolation
Build itDesign a 3-agent system (planner + researcher + writer) for producing a short report. Draw the message flow and the termination condition.
✓ CheckpointJustify a second agent by naming the coordination cost you accepted. If you cannot name it, one agent was enough.
In 2026, evaluation is the new system design. You cannot tell if a change helped by eyeballing one output — you need a golden dataset, metrics (faithfulness, relevance, RAGAS-style scoring), and a regression suite that runs on every change. Treat prompts and pipelines like production code.
Golden datasets
LLM-as-judge
Faithfulness & relevance
Regression testing
Build itWrite 10 input/expected pairs for one feature, build a simple LLM-as-judge scorer, then intentionally break the prompt and watch the score fall.
✓ CheckpointExplain why an LLM judge that agrees with you 95% of the time may still be useless, and what you would check before trusting a score.
Anything user-facing is under attack. Prompt injection can overwrite your system prompt; PII can leak; jailbreaks exist for every model. Defence is layered: input filtering, output validation, PII redaction, grounding checks, and refusal handling.
Prompt injection defence
PII redaction
Output validation
Refusal handling
Build itTry five prompt-injection attacks on one of your prompts, then patch each hole. Document which defence stopped which attack.
✓ CheckpointExplain why input filtering is not a guardrail on its own, and where you would put the check that actually holds.
Serving LLMs cheaply and fast is its own discipline: the KV cache, continuous batching (vLLM), quantization, and streaming. Understanding what makes generation slow — and that attention is memory-bound — lets you hit latency targets without burning the budget.
KV cache
Continuous batching
Quantization
Streaming & TTFT
Build itFor a target latency, reason through the levers: batch size, quantization, context length. Which one would you pull first, and what does it cost you?
✓ CheckpointExplain why batching improves throughput and hurts latency at the same time, and which one your product is actually measured on.
Most production LLM traffic is repetitive. Prompt caching, semantic caching (serve a cached answer for a similar question), and routing cheap questions to small models slash both latency and spend. Cost control is a first-class feature, not an afterthought.
Prompt / semantic caching
Model routing (small vs large)
Spend limits
Cache invalidation
Build itDesign a semantic cache: when do you return a cached answer vs regenerate? Pick a similarity threshold and reason about false hits.
✓ CheckpointExplain why a semantic cache can serve a confidently wrong answer, and what threshold decision that risk lives behind.
LLM systems fail silently — quality drifts with no error thrown. Trace every request, log prompt/response pairs with metadata, sample quality with an LLM judge, and alert when it drops. You will need those logs when debugging a regression at 2am.
Tracing
Structured logging
Quality sampling
Drift & alerts
Build itDesign a logging schema for a production LLM call: which 8 fields do you capture, and what condition fires an alert? Write it as JSON.
✓ CheckpointName what you would log to debug a bad answer three days later — and what you would refuse to log.
At scale you put a gateway in front of every model: one interface, multi-provider routing, fallbacks when a provider fails, rate limits, key management, and centralized logging and spend. It is the control plane that turns scattered API calls into a governed system.
Unified gateway
Multi-provider routing
Fallbacks & retries
Rate limiting & keys
Build itSketch a gateway: how do you route between two providers, fail over when one is down, and enforce a per-team rate limit?
✓ CheckpointExplain what a gateway gives you that calling the provider directly does not, beyond a single API surface.
The common questions before you start — how long it takes, whether to follow it in order, and how it stays current.
How long does this roadmap take?
It runs 18 stations across 3 tracks — roughly ~16h of focused learning, plus the time you spend actually building. It is self-paced, so most people work through it over a few weeks, an evening or a single station at a time.
Do I have to follow the stations in order?
The tracks are ordered so each station builds on the one before, and following them start to finish is the intended path. But every station also stands alone — if you already have the foundations, jump straight to the part you need.
Is it free?
Yes. The whole roadmap, and every handbook, lab, and challenge it links to, is free and open — no sign-up and no paywall.
How is the roadmap kept current?
It teaches the durable fundamentals of the role first, then the tooling and the AI-era shifts on top — so most of it stays relevant as individual tools churn, and it is revised as the role itself changes.
Who is this roadmap for?
Anyone stepping into or leveling up in the AI Engineer role — whether you are switching in, early-career, or a senior filling gaps. Start where you are; the ladder shows what is left.
Finished this one? 0 / 31 Roadmaps done
Explore the topic
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.