Vibe Engines Learn
Visual course · LLM internals

How LLMs actually work.

Everyone uses language models; almost no one can say what's happening inside one. This course fixes that — seven short, visual episodes that build the whole machine from the ground up: how text becomes tokens, how tokens become meaning, how attention weighs context, and how the biggest models are routed, tuned, aligned, and — for images — run entirely in reverse. Every episode is embedded here, with a hands-on lab and the deeper reference on each idea.

7 episodes ~90 min Visual / first-principles Free · no sign-up
Your progress0 / 7 complete

What you'll understand

By the end you'll have a real mental model of a language model end to end — not analogies, the actual mechanism. You'll know why a model can't count the letters in "strawberry," what an embedding really is, why attention was the breakthrough, how a trillion-parameter model runs cheaply per token, the difference between fine-tuning and LoRA, why RLHF is what makes a model helpful, and how image models generate from noise. Each idea has a companion lab so you can poke it yourself.

AttentionTokenizationEmbeddingsMeaning-spaceMixture-of-ExpertsFine-tuningLoRARLHFReward modelsDiffusion
The curriculum

Seven episodes, in order

01
Episode 1 · Attention

The Idea That Made LLMs Possible

Older models read left to right with a fading memory. Attention threw that out: every token gets to look at every other token at once and decide what matters. It's the mechanism at the heart of the transformer — and the reason the last decade of AI happened.

What you'll learn
  • Why sequence memory used to fade
  • Query, key and value — attention in plain terms
  • How self-attention weighs the whole context at once
Go deeper
The Transformers Handbook Attention Visualizer (lab) Attention Is All You Need (paper)
02
Episode 2 · Tokenization

Why AI Can't Count the R's in Strawberry

A model never sees words — it sees tokens, the sub-word pieces text is chopped into first. That single fact explains why models miscount letters, fumble rare words, and bill you the way they do. Once you see tokenization, a lot of "weird" AI behavior stops being weird.

What you'll learn
  • What a token actually is (and isn't)
  • Byte-pair encoding, in plain terms
  • Why tokens drive both cost and quirks
Go deeper
The Tokenizer (lab) Token Counter Tokenization (glossary)
03
Episode 3 · Embeddings

Turning Words Into Arrows of Meaning

Every token becomes a vector — a point in a high-dimensional meaning-space where similar ideas sit close together and directions carry meaning. Embeddings are what let a model, and a search engine, work with meaning instead of exact strings. This is the quiet foundation under RAG and semantic search.

What you'll learn
  • What an embedding really is
  • Meaning as direction + distance
  • Nearest-neighbor search over vectors
Go deeper
Embedding Space (lab) The Vector Databases Handbook Embedding (glossary)
04
Episode 4 · Mixture-of-Experts

How Trillion-Parameter Models Stay Cheap

The biggest models don't run all their weights on every token. A router sends each token to just a few specialized experts, so a model can hold enormous total capacity while only paying to run a slice of it per token. It's how frontier models got huge without getting proportionally slow.

What you'll learn
  • Dense vs sparse (MoE) layers
  • The router and top-k expert selection
  • Total vs active parameters — and why it matters
Go deeper
Mixture-of-Experts (glossary) The Distributed Training Handbook
05
Episode 5 · Fine-tuning & LoRA

Teaching a Base Model a New Trick

A base model is a generalist. Fine-tuning nudges it toward a task or a voice — and LoRA does it cheaply by training tiny adapter matrices instead of all the weights. It's the trick behind most custom models, and why you can specialize a big model on a single GPU.

What you'll learn
  • Full fine-tuning vs parameter-efficient tuning
  • How LoRA adapters work
  • When to fine-tune vs just retrieve
Go deeper
The Fine-Tuning Handbook LoRA (paper) Fine-Tuning Cost Estimator
06
Episode 6 · RLHF

What Makes a Model Helpful, Not Just Fluent

Pretraining makes a model fluent, not helpful — it will happily continue your text without answering it. Reinforcement learning from human feedback trains a reward model on human preferences and uses it to align the model toward the answers people actually want. It's the step that turned raw LLMs into assistants.

What you'll learn
  • Why a pretrained model isn't yet an assistant
  • Reward models from human preference data
  • The align-with-feedback loop, in plain terms
Go deeper
The Reinforcement Learning Handbook InstructGPT (paper) RLHF (glossary)
07
Episode 7 · Diffusion · Finale

Generating Images From Pure Noise

Image models work backwards: start from pure static and denoise, step by step, toward a picture that matches the prompt. Diffusion is a completely different generative idea from next-token prediction — and the finale ties the whole family of models together into one picture.

What you'll learn
  • Forward noising vs learned denoising
  • How a prompt steers the reverse process
  • How diffusion contrasts with autoregressive LLMs
Go deeper
Diffusion Denoise (lab) The Diffusion Models Handbook Denoising Diffusion (paper)

Understood the machine? Now build with it.

You've seen what's happening inside the model end to end. The natural next step is applying it — the hands-on RAG course turns this understanding into a shipped app, and the AI Engineer roadmap frames where it all sits.

Build a Real AI App → The AI Engineer Roadmap