A written version of the interactive roadmap above — every station, what you'll learn, and a small thing to build — laid out for reading, reference and search.
Foundations Start here
F1. The ML Lifecycle
Beginner · 45 min
A model is never "done" — it lives in a loop: collect data, train, evaluate, deploy, monitor, and retrain. MLOps is the discipline of making that loop reliable and repeatable. Everything downstream rests on one habit: reproducibility, so any result can be recreated exactly.
Skills: The train→deploy→monitor loop · Reproducibility · Environments vs artifacts · Why notebooks don't ship
Build it: Take a notebook model and list every hidden dependency that would stop a teammate reproducing your result — data snapshot, random seed, library versions, hardware.
F2. Data & Feature Versioning
Beginner · 45 min
Code is versioned; data usually is not — and that is where reproducibility dies. Versioning datasets and feature definitions means every model can be traced back to the exact inputs it saw, so "it worked last week" becomes a fact you can check, not a mystery.
Skills: Dataset versioning (DVC-style) · Feature definitions · Immutable snapshots · Data lineage basics
Build it: Snapshot a dataset, train on it, then change a few rows and re-snapshot. Prove you can reproduce the original model bit-for-bit from the first snapshot.
F3. Experiment Tracking
Beginner · 45 min
Which hyperparameters produced your best model? Without tracking, that answer is lost. Experiment tracking logs params, metrics, and artifacts for every run, so you can compare hundreds of experiments and reproduce any one of them on demand.
Skills: Logging params & metrics · Run comparison · Artifact storage · Reproducible runs
Build it: Instrument a training script to log its hyperparameters and final metrics. Run it 5 times with different settings and rank the runs by validation score.
F4. Model Registry & Versioning
Intermediate · 45 min
Once you have good models, you need a source of truth for them: which version is in staging, which is in production, what data trained it, and how to roll back. A model registry gives every model a version, a stage, and a lineage you can audit.
Skills: Model versioning · Staging → production · Lineage & metadata · Rollback
Build it: Design the metadata you would store for each registered model version — enough to answer "what is in production and how do I roll it back?" in one query.
F5. Containers & Environments
Intermediate · 45 min
"Works on my machine" is the enemy of production ML. Containers (Docker) and pinned dependencies package your code, libraries, and runtime so training and serving behave identically on a laptop, a CI runner, and a GPU cluster.
Skills: Docker for ML · Dependency pinning · Reproducible builds · GPU base images
Build it: Containerize a training script with pinned dependencies. Run it on two machines and confirm the output is identical.
F6. CI/CD for ML
Intermediate · 60 min
ML deserves the same automation as software — plus a few extras. A CI/CD pipeline runs tests, validates data, trains or promotes a model, and deploys it, so a change goes from commit to production without manual, error-prone steps.
Skills: Automated tests · Data validation gates · Model promotion · Automated deploys
Build it: Sketch a CI/CD pipeline for a model: what checks run on every commit, what gates a deploy, and what happens if data validation fails?
Pipelines Level up
T1. Training Pipelines & Orchestration
Intermediate · 60 min
A real training job is a graph of steps — ingest, validate, featurize, train, evaluate, register — that must run on a schedule, retry on failure, and be observable. Orchestrators (Airflow, Kubeflow, Prefect) turn a fragile notebook into a dependable DAG.
Skills: DAGs & orchestration · Scheduling & retries · Step isolation · Pipeline observability
Build it: Decompose one training script into a 5-step DAG. For each step, name its inputs, outputs, and what should happen when it fails.
T2. Feature Stores
Advanced · 60 min
The classic MLOps bug: features computed one way in training and another way in serving, so the model sees different inputs in production. A feature store computes features once and serves the same values to both — killing training/serving skew.
Skills: Training/serving skew · Online vs offline features · Feature reuse · Point-in-time correctness
Build it: Pick a feature (e.g. "orders in last 7 days"). Explain how it could be computed differently in training vs serving, and how a feature store prevents the mismatch.
T3. Model Packaging & Serving
Intermediate · 60 min
A trained model is useless until something can call it. Packaging wraps it behind a versioned API with a defined input/output contract; serving runs that API reliably, with health checks, autoscaling, and safe rollouts.
Skills: Model as an API · I/O contracts · Health checks · Safe rollout
Build it: Define the request/response contract for a model API. What do you return on a bad input, a timeout, or a model-not-loaded error?
T4. Batch vs Online Inference
Intermediate · 45 min
Not every prediction needs to be real-time. Batch inference precomputes predictions in bulk on a schedule (cheap, high-throughput); online inference answers per request in milliseconds (costly, low-latency). Choosing right shapes your whole architecture.
Skills: Batch scoring · Online / real-time serving · Latency vs throughput · Caching predictions
Build it: For three use cases (recommendations email, fraud check, search ranking) decide batch or online, and justify the latency and cost tradeoff.
T5. Scaling & GPUs
Advanced · 60 min
Under load, serving becomes a systems problem: autoscaling replicas, batching requests, and squeezing GPU utilization so you are not paying for idle silicon. The goal is meeting latency targets at the lowest cost per prediction.
Skills: Autoscaling · Request batching · GPU utilization · Cost per prediction
Build it: For a service at 1,000 requests/second with a 100ms budget, reason through replicas, batch size, and GPU count. Where is the bottleneck?
T6. LLMOps
Advanced · 60 min
LLMs add a loop on top of classic MLOps: prompts are versioned artifacts, evals gate releases, and fine-tuning is its own pipeline. LLMOps is MLOps plus prompt management, eval-driven deployment, and the cost/latency realities of giant models.
Skills: Prompt versioning · Eval-gated releases · Fine-tuning pipelines · Cost & latency control
Build it: Sketch the extra pipeline stages an LLM feature needs that a classic model does not — prompt versioning, eval gates, guardrails — and where each sits.
Operations Run it
P1. Monitoring & Drift
Advanced · 60 min
Models rot. The world shifts under them — user behaviour changes, upstream data changes — and accuracy quietly decays with no error thrown. Monitoring watches for data drift and prediction drift so you catch decay before your users do.
Skills: Data drift · Prediction / concept drift · Baseline distributions · Alerting on decay
Build it: Pick a feature and define a drift check: what distribution do you compare against, and what change would trigger an alert?
P2. A/B Testing & Rollouts
Advanced · 60 min
Never trust a new model blindly. Shadow it against the old one, canary it to a slice of traffic, and A/B test to prove it actually improves the metric that matters. Rollout strategy is how you ship models without betting the whole system.
Skills: Shadow deployment · Canary releases · A/B testing · Guardrail metrics
Build it: Design the rollout for a new ranking model: shadow → canary → full. What metric decides promotion, and what triggers an automatic rollback?
P3. Observability & Logging
Intermediate · 45 min
When a prediction goes wrong at 2am, logs are all you have. Trace each request, log inputs, outputs, model version, and latency, and alert on quality and error rates. Observability turns "the model is acting weird" into a debuggable timeline.
Skills: Request tracing · Prediction logging · Latency & error alerts · Debuggability
Build it: Design a logging schema for a prediction service: which 8 fields do you capture, and what condition fires an alert? Write it as JSON.
P4. Cost & Resource Optimization
Intermediate · 45 min
ML infrastructure bills add up fast — idle GPUs, oversized instances, redundant recomputation. Right-sizing compute, caching predictions, spot instances, and turning off what you are not using keeps the system affordable at scale.
Skills: Right-sizing compute · Caching & spot instances · Idle-resource cleanup · Cost attribution
Build it: Audit a serving setup for waste: where would you cache, where would you downsize, and where is a GPU sitting idle you could reclaim?
P5. Governance & Lineage
Intermediate · 45 min
In regulated or high-stakes settings you must answer: what data trained this model, who approved it, and why did it make that decision? Lineage and governance track the full provenance of every model and prediction for audit and compliance.
Skills: Model & data lineage · Audit trails · Access control · Compliance & documentation
Build it: For a model in a regulated domain, list what you would need to store to answer an auditor: training data, approvals, and decision traceability.
P6. Incident Response & Retraining
Advanced · 60 min
The loop closes here. When monitoring flags a regression, you need a runbook: roll back to a known-good version, diagnose the cause, and trigger retraining on fresh data. Automated retraining pipelines turn decay from a crisis into a routine.
Skills: Rollback runbooks · Root-cause diagnosis · Automated retraining · Closing the loop
Build it: Write an incident runbook: a model's accuracy drops 10% overnight — what are your first five steps, in order?