Vibe Engines
YouTube
LEARNING PATH · AI Engineering

ML Engineering Foundations

For developers moving from app code into machine learning.

Beginner ~2h 15m7 lessons7 steps

The core math and primitives behind modern ML, made runnable: feel how models learn, implement the functions every model relies on, then assemble them into a recommender and a retrieval pipeline.

  • Build intuition for how gradient descent trains a model
  • Implement softmax, cosine similarity and top-k ranking from scratch
  • Connect the primitives to a real recommendation system
  • See where retrieval fits in a modern ML stack
0 / 7 done · 0%
  1. LabNext up

    Gradient Descent: The Descent

    Play with how a model actually learns.

  2. Challenge

    Softmax

    The function behind every classifier head.

  3. Challenge

    Cosine Similarity

    How embeddings are compared.

  4. Challenge

    Top-K Retrieval

    Rank candidates by similarity.

  5. Paper Breakdown

    Attention Is All You Need

    See those exact primitives — similarity + softmax — assembled into the Transformer.

  6. AI System Design

    Design a Recommendation System

    Assemble the primitives into a recommender.

  7. AI System Design

    Design a RAG Pipeline

    Where retrieval fits in a modern stack.

How this path works

This path is for developers who build apps that use machine learning, such as a feed, a search box or a chat feature, and want to know what is happening inside them. It is marked Beginner: you need to read basic Python and remember what a slope and a sum are, and that is enough to start.

Machine learning engineering asks for a small vocabulary used precisely. You should be able to say what a loss is and how a gradient reduces it, why a model's raw scores get turned into probabilities, how two embeddings are compared, and why systems retrieve a shortlist before they rank it. Those ideas come up constantly: in design discussions, in debugging a model that will not train, and in interviews, which often ask you to explain or implement one of them.

The steps build on each other. Gradient descent comes first because every model on the path is trained that way. Softmax, cosine similarity and top-k retrieval follow as three short exercises, the last one reusing the second. The Transformer paper then shows the same similarity and softmax inside attention, and two system designs finish the path by placing your primitives in real products: a recommender and a retrieval pipeline for a language model.

What each step gives you

  1. Gradient Descent: The Descent

    An interactive lab: roll a ball down a loss curve by hand, then let the optimizer follow the slope, push the learning rate until training diverges, and escape a local minimum. Afterward you can state the update rule, explain why a learning rate that is too large makes the loss explode instead of shrink, and say what SGD, momentum and Adam each change.

  2. Softmax

    Your first implementation exercise turns a list of raw scores into probabilities that sum to one, with hidden tests that include very large logits. Subtracting the maximum before exponentiating is the real lesson, and it is the instinct you will need whenever exponentials meet floating-point numbers, from classifier outputs to the attention weights you meet three steps later.

  3. Cosine Similarity

    Compare two vectors by the angle between them: divide their dot product by the product of their lengths, which gives 1 for the same direction, 0 for orthogonal and -1 for opposite. This one function is what makes embeddings useful, and it prepares you to explain why two texts of very different length can still be close neighbors.

  4. Top-K Retrieval

    Put the previous function to work: score every candidate against a query, sort best first with a stable tie-break, and return the top k indices. This brute-force loop is where recommenders and search engines begin, and it raises the question the system designs at the end of the path answer: what changes when there are a hundred million candidates instead of ten?

  5. Attention Is All You Need

    Here the primitives you just wrote reappear inside the model behind modern language AI: attention scores are dot products between queries and keys, scaled and passed through softmax to become weights over the values. Read the breakdown with your own implementations open and you can explain self-attention as similarity plus softmax, then work its formula on a small example by hand.

  6. Design a Recommendation System

    A recommender built step by step: a retrieve-then-rank funnel, user and item embeddings, two-tower retrieval with approximate nearest-neighbor search, a heavier ranking model, a feature store, filters and diversity rules, and retraining on logged engagement. You see cosine similarity and top-k at the scale of millions, plus production problems such as train-serve skew that no single function can show you.

  7. Design a RAG Pipeline

    The final step moves the same retrieval into language-model products: documents are chunked and embedded, the query is embedded the same way, and the closest matches are reranked and placed into the prompt so the model answers from evidence. It prepares you to explain where retrieval sits in a modern stack, and why the recall versus precision trade-off shows up in every system on this path.

A study plan

  1. Session 1

    Play The Descent twice, once by hand and once letting the optimizer run. Then write the update rule from memory, try three learning rates on paper (one that crawls, one that converges, one that explodes), and explain each outcome in a sentence.

  2. Session 2

    Solve the Softmax and Cosine Similarity challenges without opening the hints. If a hidden test fails, read the common-mistakes section before retrying, and finish by explaining out loud why subtracting the maximum does not change softmax's output.

  3. Session 3

    Solve Top-K Retrieval, then rank five short sentences against a query by hand, using made-up two-dimensional vectors. Seeing the scores and the tie-break on paper makes every choice in your code obvious.

  4. Session 4

    Read the Attention Is All You Need breakdown and compute attention for three tokens by hand, using your softmax function to check the arithmetic. Mark where the similarity score and the softmax appear in the formula.

  5. Session 5

    Read Design a Recommendation System and Design a RAG Pipeline back to back. Draw both funnels side by side and label where gradient descent, softmax, similarity and top-k each appear in them.

Questions about this path

What math do I need for machine learning engineering?

For engineering work, intuition beats proofs. Linear algebra gives you vectors, dot products and matrix multiplication, which cover embeddings and attention. Calculus gives you derivatives as slopes, which is all gradient descent needs. Probability explains softmax, sampling and why outputs are distributions, and basic statistics lets you read evaluation results without fooling yourself. You can learn each piece alongside code, which is how this path introduces them: every idea arrives as a lab or a short function with tests.

Can a web developer learn machine learning without a math degree?

Yes, for applied work. The core ideas behind modern ML systems fit in short functions: a gradient step, a softmax, a similarity score, a ranking. Implementing them yourself, as this path asks, builds more usable understanding than reading formulas. A math degree matters more for research, for designing new training methods, or for the most theoretical papers. If you later need more depth, add linear algebra and probability as concrete questions come up rather than all at once up front.

What is the difference between machine learning and AI engineering?

Machine learning engineering centers on training models from data and running them in production: datasets, features, training pipelines, evaluation and retraining. AI engineering, in the sense used today, usually means building applications on top of large pretrained models, with prompting, retrieval, agents and evals. The foundations overlap. Embeddings, similarity search, softmax and ranking show up in both, which is why this path works as a first step toward either the Become an AI Engineer path or the Build an LLM From Scratch path.

What should I learn after gradient descent and embeddings?

Choose by what you want to build. To ship features on top of language models, move to the Become an AI Engineer path, which takes retrieval, agents, evaluation and serving into production detail. To understand how a language model works inside, the Build an LLM From Scratch path goes from tokenization through attention, the training loss and preference tuning. Either way, keep implementing small pieces yourself, because that habit is what makes the next layer of material readable.

From app code into ML

You don't need a research background to build with machine learning, but a handful of core ideas repay understanding directly — and this path makes them runnable. Feel how a model learns by descending a gradient, implement the softmax every model head relies on, then the similarity and ranking that power search and recommendations.

Each concept is a lab you drive or a function you implement with tests, not a formula to memorise. By the end the rest of AI engineering — a recommender, a retrieval pipeline, an embedding search — reads as combinations of primitives you've built yourself.

← All learning paths