A written version of the interactive roadmap above — every station, what you'll learn, and a small thing to build — laid out for reading, reference and search.
Foundations Start here
F1. Python & ML Basics
Beginner · 60 min
Every AI engineer builds on Python and a working feel for machine learning. Arrays and tensors, training vs inference, features and loss, overfitting — you do not need to invent models, but you need the vocabulary to reason about the ones you use.
Skills: Python for AI · Arrays / tensors · Train vs inference · Loss & evaluation intuition
Build it: Load a small dataset, train a simple classifier with scikit-learn, and read its accuracy. Then break it — shuffle the labels — and watch accuracy collapse to chance.
✓ Checkpoint: Explain what a model actually learned when accuracy collapses to chance after you shuffle the labels — and why that is the most informative experiment on this station.
F2. How LLMs Work
Beginner · 60 min
An LLM predicts the next token, over and over. Under the hood is the Transformer — self-attention lets every token look at every other token at once. Understanding tokens, attention, and sampling (temperature, top-p) demystifies everything the model does downstream.
Skills: Tokens & next-token prediction · Self-attention · Context windows · Temperature & sampling
Build it: Tokenize a paragraph with a real tokenizer and count the tokens. Then generate the same prompt at temperature 0 and 1.0 five times each, and compare how much the output varies.
✓ Checkpoint: Explain why the same prompt at temperature 0 can still be wrong in the same way every time, and what that tells you about where the error lives.
F3. Embeddings & Vectors
Beginner · 45 min
Turn text into a vector and meaning becomes geometry: similar things sit close together. Embeddings and cosine similarity are the engine under semantic search, recommendations, and retrieval — the single most useful representation in applied AI.
Skills: Embeddings · Cosine similarity · Nearest-neighbour search · Semantic vs keyword match
Build it: Embed 20 short sentences, then for a query sentence rank the rest by cosine similarity. Check whether the top matches actually mean the same thing.
✓ Checkpoint: Explain why two sentences that share no words can sit close together in embedding space, and name a case where that closeness misleads you.
F4. Prompt Engineering
Beginner · 45 min
The prompt is your primary interface to the model. System vs user roles, clear instructions, few-shot examples, and asking the model to reason before answering are the highest-leverage, lowest-effort levers you have on quality.
Skills: System / user roles · Few-shot examples · Instruction clarity · Reason-then-answer
Build it: Take one task and write it three ways: zero-shot, few-shot, and reason-then-answer. Run each on 10 inputs and tally which is most reliable.
✓ Checkpoint: Take a prompt that failed and say whether the fix belongs in the instruction, the examples, or the context you supplied — “write a better prompt” is not an answer.
F5. Tokens, Context & Cost
Intermediate · 30 min
Every token costs money and latency, and every model has a finite context window. Knowing how text becomes tokens, what fits in the window, and how pricing works is the difference between a demo and something you can afford to run.
Skills: Tokenization · Context-window budgeting · Pricing math · Model size vs cost
Build it: Estimate the token count and dollar cost of a long RAG prompt before sending it. Then trim it 30% and re-estimate — how much did you save?
✓ Checkpoint: Compute the cost of one request from its token count and price per million, then say which part of the prompt you would cut first and what it would break.
F6. Structured Output & Tools
Intermediate · 45 min
To wire an LLM into software you need reliable structure, not prose. JSON mode, output schemas, and function/tool calling let the model return data your code can trust — and let it call your functions with well-typed arguments.
Skills: JSON mode · Output schemas · Function / tool calling · Validation & retries
Build it: Design a tool schema for a "get_weather(city, unit)" function, then prompt the model to call it correctly on 10 varied requests. Count schema violations.
✓ Checkpoint: Explain why asking a model for JSON is not the same as guaranteeing JSON, and what your code does on the day it returns prose instead.
Build Level up
T1. Retrieval-Augmented Generation
Intermediate · 60 min
RAG lets a model answer from your documents instead of only its frozen memory: retrieve the most relevant chunks, then generate grounded in them. It is the backbone of every "chat with your docs" product and the main tool for cutting hallucination.
Skills: Chunking · Retrieve-then-generate · Grounding & citations · Hallucination reduction
Build it: Build a minimal RAG loop: embed a document, retrieve top-k chunks for a question, and prompt the model to answer using only those chunks, with citations.
✓ Checkpoint: Explain why a retrieval system can return the correct document and still produce a wrong answer, and which half of the pipeline you would investigate first.
T2. Vector Databases
Intermediate · 60 min
At scale you need fast nearest-neighbour search over millions of vectors. Vector databases use approximate indexes (HNSW, IVF) to trade a little accuracy for huge speed, and pair with chunking strategies and re-rankers to make retrieval actually good.
Skills: ANN indexes (HNSW / IVF) · Chunking strategy · Hybrid search · Re-ranking
Build it: Index a few thousand chunks in a vector DB and compare exact vs approximate search: measure recall and latency at different index settings.
✓ Checkpoint: Explain what approximate means in approximate nearest neighbour, and what you traded to get the speed.
T3. Agents & Tool Use
Advanced · 60 min
An agent thinks, acts, observes, and repeats — the ReAct loop. Give a model tools plus a reasoning loop and it can pursue multi-step goals: search, call APIs, read results, and adjust. Most of the hard work is tool schemas, memory, and loop control.
Skills: ReAct loop · Tool schemas · Short/long-term memory · Loop termination & recovery
Build it: Build a 2-tool agent (search + calculate). Run it on 5 multi-step questions and log where it loops, hallucinates a tool call, or stops too early.
✓ Checkpoint: Explain why an agent that never stops is a design failure rather than a bug, and name the stopping conditions yours actually has.
T4. Reasoning & Chain-of-Thought
Intermediate · 45 min
Ask a model to show its working and multi-step accuracy jumps. Chain-of-thought, self-consistency (sample many chains, vote), and modern reasoning models all rest on one idea: give the model room to think before it commits to an answer.
Skills: Chain-of-thought · Self-consistency · When reasoning helps · Reasoning models
Build it: Take 10 word problems. Answer each directly, then with "let's think step by step". Measure the accuracy gap.
✓ Checkpoint: Explain what chain-of-thought buys you and what it costs per request — and name a task where it makes the answer worse.
T5. Fine-Tuning & LoRA
Advanced · 60 min
When prompting and RAG are not enough, fine-tune. LoRA freezes the base model and trains tiny adapters — cheap, fast, and swappable — while preference tuning (DPO) aligns behaviour. The key skill is knowing when fine-tuning beats retrieval, and when it does not.
Skills: RAG vs fine-tuning · LoRA / QLoRA adapters · Preference tuning (DPO) · Dataset curation
Build it: Sketch a decision: for a given task, argue whether RAG, fine-tuning, or both is right — and what data you would need for the fine-tune.
✓ Checkpoint: Say whether your problem is the model not KNOWING something or not BEHAVING a certain way, and defend the choice that follows.
T6. Multi-Agent Orchestration
Advanced · 60 min
Some problems are better split across specialised agents — a planner that decomposes work, workers that execute, and handoffs between them. Orchestration is about coordination: routing, shared state, and stopping the whole system from looping or drifting.
Skills: Planner / worker patterns · Agent handoffs · Shared state · Failure isolation
Build it: Design a 3-agent system (planner + researcher + writer) for producing a short report. Draw the message flow and the termination condition.
✓ Checkpoint: Justify a second agent by naming the coordination cost you accepted. If you cannot name it, one agent was enough.
Production Ship it
P1. Evaluation & LLM-as-Judge
Advanced · 90 min
In 2026, evaluation is the new system design. You cannot tell if a change helped by eyeballing one output — you need a golden dataset, metrics (faithfulness, relevance, RAGAS-style scoring), and a regression suite that runs on every change. Treat prompts and pipelines like production code.
Skills: Golden datasets · LLM-as-judge · Faithfulness & relevance · Regression testing
Build it: Write 10 input/expected pairs for one feature, build a simple LLM-as-judge scorer, then intentionally break the prompt and watch the score fall.
✓ Checkpoint: Explain why an LLM judge that agrees with you 95% of the time may still be useless, and what you would check before trusting a score.
P2. Guardrails & Safety
Intermediate · 60 min
Anything user-facing is under attack. Prompt injection can overwrite your system prompt; PII can leak; jailbreaks exist for every model. Defence is layered: input filtering, output validation, PII redaction, grounding checks, and refusal handling.
Skills: Prompt injection defence · PII redaction · Output validation · Refusal handling
Build it: Try five prompt-injection attacks on one of your prompts, then patch each hole. Document which defence stopped which attack.
✓ Checkpoint: Explain why input filtering is not a guardrail on its own, and where you would put the check that actually holds.
P3. Inference Serving & Latency
Advanced · 60 min
Serving LLMs cheaply and fast is its own discipline: the KV cache, continuous batching (vLLM), quantization, and streaming. Understanding what makes generation slow — and that attention is memory-bound — lets you hit latency targets without burning the budget.
Skills: KV cache · Continuous batching · Quantization · Streaming & TTFT
Build it: For a target latency, reason through the levers: batch size, quantization, context length. Which one would you pull first, and what does it cost you?
✓ Checkpoint: Explain why batching improves throughput and hurts latency at the same time, and which one your product is actually measured on.
P4. Caching & Cost Control
Intermediate · 45 min
Most production LLM traffic is repetitive. Prompt caching, semantic caching (serve a cached answer for a similar question), and routing cheap questions to small models slash both latency and spend. Cost control is a first-class feature, not an afterthought.
Skills: Prompt / semantic caching · Model routing (small vs large) · Spend limits · Cache invalidation
Build it: Design a semantic cache: when do you return a cached answer vs regenerate? Pick a similarity threshold and reason about false hits.
✓ Checkpoint: Explain why a semantic cache can serve a confidently wrong answer, and what threshold decision that risk lives behind.
P5. Observability & Monitoring
Intermediate · 45 min
LLM systems fail silently — quality drifts with no error thrown. Trace every request, log prompt/response pairs with metadata, sample quality with an LLM judge, and alert when it drops. You will need those logs when debugging a regression at 2am.
Skills: Tracing · Structured logging · Quality sampling · Drift & alerts
Build it: Design a logging schema for a production LLM call: which 8 fields do you capture, and what condition fires an alert? Write it as JSON.
✓ Checkpoint: Name what you would log to debug a bad answer three days later — and what you would refuse to log.
P6. LLM Gateway & Routing
Advanced · 60 min
At scale you put a gateway in front of every model: one interface, multi-provider routing, fallbacks when a provider fails, rate limits, key management, and centralized logging and spend. It is the control plane that turns scattered API calls into a governed system.
Skills: Unified gateway · Multi-provider routing · Fallbacks & retries · Rate limiting & keys
Build it: Sketch a gateway: how do you route between two providers, fail over when one is down, and enforce a per-team rate limit?
✓ Checkpoint: Explain what a gateway gives you that calling the provider directly does not, beyond a single API surface.