Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path →
18 stations. 3 tracks. The build-and-train-models path, not just features on top: from the math and PyTorch, deep-learning fundamentals and reading papers, through data pipelines, training dynamics, distributed training and scaling laws, to post-training, evaluation, efficiency and the research craft.
18 stations on 3 ladders, in order — each rung is written assuming the one above it. Mark them off as you go; your progress is saved in this browser.
6 rungs · ~6h · ends at Reproducing Results.
F1 · Intermediate · 35 min
A research engineer builds and trains models and the systems that train them — sitting between the researcher who has the idea and the infrastructure that runs it. Learn how the role differs from an AI/ML engineer (who mostly builds features on existing models) and from a pure research scientist (who focuses on the ideas): the research engineer makes experiments real, fast, and reproducible. This is one of the most in-demand and hard-to-automate technical roles.
Build itA researcher has a promising idea but no working code. Explain what a research engineer does to turn it into a run you can trust.
✓ CheckpointExplain what a research engineer does that a research scientist does not, and why an experiment nobody can re-run is not a result.
F2 · Intermediate · 55 min
You don’t need to be a mathematician, but research engineering rests on real math: linear algebra (vectors, matrices, the operations behind every layer), calculus and the chain rule (how gradients and backprop work), and probability (distributions, likelihood, the basis of loss functions). Learn enough to read the equations in a paper and understand what the code is actually doing, not just to copy it.
Build itBackprop is "just the chain rule." Explain in plain terms how the chain rule turns a loss into a gradient for an early layer.
✓ CheckpointRead a paper's central equation aloud in plain language. If you can only recognise the symbols, you cannot yet check the claim.
F3 · Intermediate · 60 min
PyTorch is the research engineer’s primary tool, and deep-learning fundamentals are the material. Learn the core: tensors and autograd, building models as modules, the training loop (forward, loss, backward, step), and how optimizers and learning rates actually behave. Build a model from scratch so you understand every line — the intuition you get from doing it by hand is what lets you debug the hard cases later.
Build itWrite, in pseudocode, the minimal PyTorch training loop and name what each of the four core steps does.
✓ CheckpointExplain what autograd is actually building while your forward pass runs, and why that is what makes backprop automatic.
F4 · Advanced · 55 min
The Transformer is the architecture behind modern AI, and a research engineer must understand it to the bone. Learn self-attention (why it works, the math), multi-head attention, positional encoding, the full block, and how the pieces compose into an LLM. Implement attention yourself — the difference between using a Transformer and understanding one is exactly the difference between an applied engineer and a research engineer.
Build itExplain why attention is O(n²) in sequence length, and what that means for training on long contexts.
✓ CheckpointDerive the shape of the attention matrix from sequence length and head count, and say where the quadratic cost comes from.
F5 · Intermediate · 45 min
Research moves through papers, and reading them well is a core skill. Learn to read efficiently (abstract → figures → method → results), to extract the actual contribution from the framing, to judge a paper critically (is the comparison fair, are the results significant?), and to keep up with a fast field without drowning. A research engineer who reads papers sharply turns the frontier into things they can build.
Build itA paper claims a big win. List the three things you’d check before believing the result is real and reproducible.
✓ CheckpointTake a recent paper and state its real contribution in one sentence, separate from what the abstract claims.
F6 · Advanced · 55 min
The defining research-engineer skill is turning a paper into working, verified code — reproducing a result from scratch. Learn to translate a method into an implementation, to match the setup precisely, to debug the inevitable gaps between paper and reality, and to know when your reproduction is genuinely correct. Reproducing results is the best way to learn deeply, and it’s exactly what the role is paid to do.
Build itYour reimplementation gets worse numbers than the paper. List the usual suspects you’d check, in order.
✓ CheckpointName a reproduction gap that was the paper's fault and one that was yours, and explain how you told them apart.
6 rungs · ~6h · ends at Debugging Training.
T1 · Advanced · 50 min
Models are only as good as their data, and building the data pipeline is a huge part of real research engineering. Learn to source, clean, deduplicate, filter, and tokenize data at scale, why data quality often matters more than model tweaks, and how to build pipelines that feed training efficiently. The unglamorous data work is frequently where the actual results come from.
Build itA model underperforms despite a good architecture. Explain why the data pipeline is the first place a research engineer looks.
✓ CheckpointExplain why deduplication changes model quality more than most architecture tweaks, and what a duplicate does during training.
T2 · Advanced · 55 min
Getting a model to train well is an art built on understanding. Learn the dynamics: how learning rate schedules, batch size, initialization, and normalization interact; what loss curves tell you; and the common failure modes — divergence, instability, plateaus. A research engineer reads a training run like a doctor reads vitals, diagnosing what’s wrong and what to change.
Build itA training run’s loss spikes and never recovers. Walk through the likely causes and what you’d change first.
✓ CheckpointLook at a loss curve that plateaus and one that diverges, and say which hyperparameter you would reach for in each case.
T3 · Advanced · 60 min
Real models don’t fit on one GPU, so scaling training across many is core research-engineering craft. Learn the parallelism strategies — data, tensor, pipeline, and sharding (ZeRO/FSDP) — what each trades off, how communication between GPUs becomes the bottleneck, and how to actually run a multi-GPU or multi-node job. This is where research engineering meets serious systems work.
Build itA model is too big for one GPU’s memory. Explain the difference between data parallelism and sharding for solving that.
✓ CheckpointExplain the difference between data, tensor and pipeline parallelism in terms of what gets split, and which one communication cost bites first.
T4 · Advanced · 45 min
Scaling laws are one of the most important ideas in modern ML: model performance improves predictably with model size, data, and compute, and there’s a compute-optimal balance between them. Learn what the laws say (and their limits), how to use them to plan experiments and allocate a compute budget, and why they reshaped how frontier models are built. Reasoning in terms of scaling is central to research-engineering judgment.
Build itYou have a fixed compute budget. Explain how scaling-law thinking guides the trade-off between a bigger model and more data.
✓ CheckpointGiven a fixed compute budget, explain how you would split it between model size and tokens — and where the scaling laws stop being reliable.
T5 · Intermediate · 40 min
Research is a stream of experiments, and without discipline it becomes chaos you can’t learn from. Learn to track experiments rigorously — configs, metrics, and artifacts versioned so every run is reproducible and comparable, to change one thing at a time, and to build the infrastructure that lets a team run and compare hundreds of experiments. Good experiment hygiene is what turns activity into knowledge.
Build itTwo runs give different results and no one recorded what differed. Describe the tracking that would have made the comparison meaningful.
✓ CheckpointExplain why changing two things between runs makes both results uninterpretable, however tempting it is.
T6 · Advanced · 50 min
Training bugs are uniquely nasty — silent, expensive, and often only visible in the results. Learn to debug systematically: verify data and shapes, check gradients and numerics (NaNs, overflow), isolate on small scales before big runs, and reason about whether a bad result is a bug or a genuine finding. The research engineer who debugs training well saves enormous time and compute, and that skill is deeply hard to automate.
Build itYour loss is NaN after a few steps. List the systematic checks you’d run, cheapest first.
✓ CheckpointDescribe how you would tell a genuine finding from a bug in your data loader, before you write it up.
6 rungs · ~6h · ends at The Research Engineer Career.
P1 · Advanced · 55 min
A pretrained model is raw capability; post-training makes it useful and aligned. Learn the modern stack: supervised fine-tuning, RLHF and its successors (DPO and beyond), instruction tuning, and how these shape a model’s behavior. Understand what each stage does, the data it needs, and the trade-offs — post-training is where much of the frontier research and the practical value now lives.
Build itA base model is capable but unhelpful and unsafe. Explain what post-training stages you’d apply and in what order.
✓ CheckpointExplain what DPO removes from the RLHF pipeline, and what you give up along with it.
P2 · Advanced · 50 min
You can’t improve what you can’t measure, and evaluating models honestly is one of the hardest research problems. Learn benchmark design and its pitfalls (contamination, gaming, benchmarks that don’t measure what matters), how to build evals that actually track the capability you care about, and why "the number went up" can be misleading. Rigorous, skeptical evaluation is a defining mark of a good research engineer.
Build itA model tops a benchmark but feels worse in use. List the ways a benchmark can lie and how you’d evaluate more honestly.
✓ CheckpointExplain how benchmark contamination inflates a number, and what you would check before believing a reported score.
P3 · Advanced · 50 min
Frontier work is bounded by compute and memory, so efficiency is core craft. Learn the techniques: mixed precision, quantization, distillation, activation checkpointing, efficient attention, and kernel-level optimization. Understand the trade-offs between speed, memory, and quality, and how to get more capability from a fixed budget. Efficiency is often what turns a research idea into something that can actually be trained or served.
Build itA model trains but is too expensive to serve. List three efficiency techniques and what each trades away.
✓ CheckpointState the accuracy cost of a quantization you have actually run, with a number rather than “minimal”.
P4 · Advanced · 45 min
Beyond skills, research is a way of working: forming a clear hypothesis, designing an experiment that could actually disprove it, running it cleanly, and reasoning honestly about what the result means. Learn to avoid the traps — fooling yourself, chasing noise, confirmation bias — and to iterate from a question to a real finding. This scientific discipline is what separates research from tinkering, and AI can’t supply it.
Build itAn experiment "worked." Describe how you’d check whether it’s a real finding or you fooled yourself.
✓ CheckpointState a current hypothesis and the experiment that would disprove it. If no result would change your mind, it is not a hypothesis.
P5 · Advanced · 45 min
A result in a notebook isn’t impact — the research engineer often bridges to production, turning a working experiment into a model that ships. Learn what changes at that boundary: robustness, serving efficiency, monitoring, and the collaboration with applied and infra teams. Being able to take research all the way to a deployed model is a rare, high-value combination and increasingly what the role is asked to do.
Build itYour research model beats the current one offline. Describe what you’d validate before it could replace production.
✓ CheckpointExplain what has to be true of a research model before it can be served, beyond the metric it was selected on.
P6 · Advanced · 40 min
The path branches: deeper into research (toward research scientist), deeper into systems (training infrastructure, ML systems), or into applied leadership. Learn where the role goes, how it differs from and connects to the applied AI-engineer track, what a strong portfolio looks like (reproductions, a real contribution, open-source work), and how the field’s pace makes continuous learning the job. Decide the direction deliberately and build the evidence for it.
Build itWrite the six-month plan that moves you from reproducing others’ work to making one small original contribution.
✓ CheckpointExplain how the research-engineer and applied AI-engineer tracks differ in what they are measured on.
The common questions before you start — how long it takes, whether to follow it in order, and how it stays current.
It runs 18 stations across 3 tracks — roughly ~18h of focused learning, plus the time you spend actually building. It is self-paced, so most people work through it over a few weeks, an evening or a single station at a time.
The tracks are ordered so each station builds on the one before, and following them start to finish is the intended path. But every station also stands alone — if you already have the foundations, jump straight to the part you need.
Yes. The whole roadmap, and every handbook, lab, and challenge it links to, is free and open — no sign-up and no paywall.
It teaches the durable fundamentals of the role first, then the tooling and the AI-era shifts on top — so most of it stays relevant as individual tools churn, and it is revised as the role itself changes.
Anyone stepping into or leveling up in the ML Research Engineer role — whether you are switching in, early-career, or a senior filling gaps. Start where you are; the ladder shows what is left.
See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.