The big idea

What makes a tutor "adaptive"?

Bolt an LLM onto a fixed list of practice problems and you've built a chatbot with homework attached — every student gets the same order, the same explanations, no matter what they already know or what specifically confused them. What actually makes a tutor adaptive isn't the LLM at all — it's a model of what this learner knows, driving what to ask next and how to explain it.

We'll build the pieces most "AI tutor" demos skip: a per-skill mastery model, a grading path that reaches for an exact checker before it ever reaches for an LLM, explanations grounded in a vetted curriculum instead of free generation, and hints that deliberately don't just hand over the answer.

How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the diagram grow, hover the boxes, and at the end wipe the mastery model and cut curriculum grounding to see what degrades and what breaks. Hit Begin.

Step 1 · The baseline

A fixed list of problems

Simplest version: every student gets problems in the same fixed order, chapter by chapter. What actually goes wrong with this for real learners?

Design decision: Every student sees the same fixed problem order. What's the core failure?

The call: It ignores what each individual student already knows and where they're actually struggling, so it's simultaneously too slow for some students and too fast for others. — A fixed order treats every learner identically regardless of their actual knowledge state. A student who has already mastered a skill is bored repeating it; a student who hasn't mastered a prerequisite is pushed forward into material built on a foundation they don't have yet.

A fixed order can't be right for every learner at once — it's simultaneously too slow for students who've already mastered a skill and too fast for students missing a prerequisite. Real personalization needs the system to actually know what this specific student knows.

Adaptivity requires state: You cannot personalize what to teach next without tracking, per skill, per student, some estimate of current mastery. Everything that follows in this design exists to build, use, and update that estimate.

Step 2 · The mastery model

Track what this student actually knows

You need to pick the next problem to be neither trivially easy nor hopelessly hard for THIS student, on THIS skill, right now. What decides that?

Design decision: How does the system decide the right next problem for a specific student's current level?

The call: Maintain a per-skill mastery estimate (e.g. a Bayesian Knowledge Tracing-style probability) updated after every attempt, and select the next problem to sit in the student's "zone of proximal difficulty" — challenging but achievable given their current estimate. — A mastery model accumulates evidence over many attempts into a probabilistic estimate per skill, and problem selection targets difficulty just above what the student can already reliably do — hard enough to stretch them, not so hard it's just frustration, and not so easy it wastes their time.

Maintain a per-skill mastery model — a probabilistic estimate (Bayesian Knowledge Tracing or similar) that updates after every attempt. The Problem Selector uses it to target each student's zone of proximal difficulty: challenging enough to be worth doing, achievable enough not to be discouraging.

The mastery model is the product: Strip away the LLM entirely and a working mastery model plus a good problem bank is already a legitimate adaptive tutor (this is roughly how Duolingo's core loop and classic Intelligent Tutoring Systems work) — the LLM, added later, makes it better at EXPLAINING, not at knowing what to ask.

Step 3 · Grading: symbolic first, LLM second

Don't use an LLM to check 2+2

A student submits an answer — sometimes a number, sometimes code, sometimes a written explanation. Should everything route through the LLM for grading?

Design decision: Answers range from "42" to a paragraph explaining a concept. Should an LLM grade all of it?

The call: Route structured answers (a number, an equation, code with test cases) to a deterministic symbolic/exact checker, and reserve the LLM only for genuinely open-ended answers (a written explanation, a proof sketch) where no exact-match check is possible. — A symbolic checker for "does this equal 42" or "do these unit tests pass" is instant, deterministic, and free of any grading-inconsistency risk. The LLM is reserved for exactly the cases that NEED judgment — evaluating whether a written explanation demonstrates real understanding — where no exact check exists.

Build a hybrid grader: structured answers (numeric, symbolic math, code with test cases) go to a deterministic exact checker — fast, free, perfectly consistent. Only genuinely open-ended answers (written explanations, proofs, reasoning) are routed to the LLM, which is the one grading tool actually suited to judgment calls.

Use the cheapest tool that's actually correct: This mirrors a broader systems principle: reach for a deterministic, symbolic solution whenever one exists, and reserve the expensive, probabilistic model for the cases that genuinely require judgment. It's faster, cheaper, and more trustworthy than routing everything through an LLM by default.

Step 4 · Close the loop

Every grade updates the mastery model

A grading result — right, wrong, partially right — is only useful if it actually changes what happens next.

Every graded attempt feeds back into the mastery model immediately: a correct answer nudges that skill's estimate up, an incorrect one nudges it down (and, ideally, flags a related prerequisite skill to revisit). The next problem the Selector picks reflects the updated estimate — this closes the loop between "how did the student just do" and "what should they see next," which is the actual mechanism of adaptivity, not a one-time setup step.

A loop, not a pipeline: It's tempting to think of tutoring as a one-way pipeline: select → grade → done. The real system is a closed loop — grade feeds mastery, mastery feeds selection, selection produces the next attempt to grade — running continuously for the length of the session and beyond.

Step 5 · Ground the explanations

Don't let the LLM freely teach facts

When a student gets something wrong, they need an explanation — and ideally one tailored to their specific misconception. Should the LLM generate that explanation purely from its own training?

Design decision: The LLM explains a concept the student got wrong. Generate freely, or ground it in something?

The call: Retrieve the relevant explanation from a vetted Curriculum Knowledge Base (RAG) and have the LLM tailor its delivery to the student's specific mistake, grounded in that retrieved, syllabus-accurate content. — Grounding constrains what the LLM is allowed to assert as fact to vetted, curriculum-accurate source material, while still letting it do what LLMs are genuinely good at: adapting HOW the explanation is delivered to this student's specific error, in natural, conversational language.

Retrieve relevant content from a vetted Curriculum Knowledge Base (RAG) before generating an explanation, and constrain the LLM to ground its answer in that retrieved material — it can still tailor the phrasing, examples, and framing to the student's specific mistake, but the underlying facts come from vetted source material, not free generation.

Grounding is a correctness feature, not a nice-to-have: A wrong hint taught confidently is worse than a slow one — it actively teaches misinformation to someone who trusts the system to know better. RAG-style grounding is how you get the LLM's language flexibility without inheriting its raw hallucination risk on factual content.

Step 6 · Hints, not answers

The Socratic gate

A stuck student asks "just tell me the answer." Should the tutor comply immediately?

Design decision: A struggling student directly asks for the answer. What should the tutor do?

The call: Escalate through a series of increasingly specific Socratic hints first — pointing at the relevant concept, then the specific step, then (only if still stuck after genuine attempts) the direct answer with a full explanation. — A hint ladder gives the student every chance to have the "aha" themselves first, which is where durable learning happens, while still guaranteeing they're never permanently stuck — the direct answer remains available as the last rung, not the first response.

Build a hint ladder: the first response to "I'm stuck" is a pointer toward the relevant concept, not the answer. Further requests escalate — a more specific hint, then the exact next step — and only after genuine attempts have been made does the tutor give the full direct answer with explanation. This is a deliberate pedagogical gate, not a capability limitation.

Pedagogy is a product requirement, not a prompt-engineering afterthought: "Don't just answer" has to be an explicit, designed system behavior — left to its own devices, an LLM optimized to be maximally helpful will happily just give the answer, which is the wrong optimization target for a tutor whose actual job is durable learning, not task completion.

Step 7 · Remember across sessions

Learning happens over months, not one sitting

A student logs back in three weeks later. Should the tutor start from scratch, or remember that they were shaky on fractions and haven't practiced since?

Maintain a long-term Learner Profile, separate from any single session's working state — persistent mastery trends, known weak spots, and a forgetting-curve estimate per skill (skills genuinely decay without practice). The Problem Selector consults this profile to bring back a skill for review right around when it's predicted to be forgotten — spaced repetition — rather than only reacting to what happens inside today's session.

Session memory vs. learner memory: This mirrors the short-term/long-term memory split in any conversational agent system, applied to pedagogy specifically: the working session needs fast, in-context state, while the durable learner profile is what makes the system actually get smarter about this individual over weeks and months, not just within one sitting.

Step 8 · The sharp edges

Cost, latency, and staying on-curriculum

Hint generation is on the interactive path — a student waiting 8 seconds for a hint is a bad experience. Mastery-model and profile updates don't need to be instant. And an LLM asked an off-topic or inappropriate question needs to stay in its lane.

Split by latency requirement: hint/explanation generation uses a fast, cheap model tuned for low latency, since it's directly in the student's interactive loop; mastery-model recomputation and profile updates can run slightly deferred/batched, since a few seconds of lag there is invisible to the student. Layer guardrails on the LLM path specifically for the tutoring context: refuse to just do the student's homework verbatim, redirect off-curriculum or inappropriate requests, and never let a jailbreak attempt bypass the hint-ladder's "don't just give the answer" gate.

Design for the unhappy path: Mastery store down → degrade to a sane default curriculum, don't hard-fail. Grounding cut → risk of confidently wrong teaching, which is why it's never optional. Slow hints → split latency-sensitive from latency-tolerant work. A tutor that only works for a cooperative student asking on-topic questions in perfect conditions is a demo.

You did it

You just designed an adaptive AI tutor.

• A —
• T — h
• G — r
• E — v
• E — x
• H — i
• A —
• L — a