LEARNING PATH · AI Engineering

AI for Product Managers

For PMs shipping AI features who want to reason about them without writing code.

Beginner ~5h12 lessons14 steps

A concept-first path for product managers: the AI stack, the build decisions (RAG vs fine-tuning, big vs small model), the cost and quality levers, and the risk surface — paired with interactive calculators so you can pressure-test a plan in a meeting, not just nod along.

Speak the AI stack credibly with your engineers
Make the build calls: RAG vs fine-tuning, big vs small model
Estimate inference cost and reason about the quality bar
Understand the risk surface — injection, leaks, and the EU AI Act

0 / 12 done · 0%

How this path works

This path is for product managers, group PMs and product-minded founders who are now accountable for AI features but do not write the code. You will not build a model here. The goal is narrower and more useful: to hold your own in the meeting where an engineer says "we could fine-tune it" or "the eval looks fine" and you need to know which questions to ask next.

What a PM is actually tested on in AI work is judgment about trade-offs. Which model size fits the latency and cost budget? Does the feature need retrieval, fine-tuning, or just a better prompt? How will anyone know the output is good enough to ship, what does each user request cost at scale, and what happens when someone attacks the feature or a regulator asks about it? Those are product decisions with technical inputs, and nobody can make them for you.

The order follows the life of a feature. You start with the map and the prompting basics, make the build decisions, look at what powers real products, price the plan, define quality, and finish with the risk surface. Each cost step has an interactive calculator, so you can change an assumption and watch the bill move instead of taking a number on faith.

What each step gives you

AI for Product Managers
A no-jargon tour of where AI helps product work and where it does not, built around three firm rules: never let a model invent user data, keep prioritization human, and protect confidential roadmap material. It ends with five workflows you can try this week, such as drafting a PRD from your own decisions, and a spot-the-danger exercise.
The Prompting Handbook
You learn the CRISP framework (context, role, instruction, specifics, proof) and see the same task fail and succeed with a bad and a good prompt. For a PM it answers a practical question: why does the demo work on Tuesday and drift on Friday? It also gives you a vocabulary for writing a clear prompt spec in a ticket.
LLM vs SLM
This is the model-size decision framed by deployment constraints rather than raw capability: cloud versus on-device, latency, privacy and price. It covers the routing pattern in which a small model handles most traffic and escalates only hard cases, plus a keyboard-app scenario. After it you can challenge a proposal to send every request to the biggest model.
RAG vs Fine-Tuning
The two usual answers to "make the model know our stuff" turn out to change different things: retrieval supplies knowledge at query time, while fine-tuning shapes behavior and style. A worked customer-support bot shows why teams often combine both. You will be able to ask whether a gap is about missing facts or about tone before anyone starts a training run.
Design ChatGPT
An interactive walkthrough of what sits underneath a chat product: the edge, the conversation store, a context rebuilt on every turn, a model router and prefix caching. You do not need to follow every box. It shows why latency, cost and memory limits are architectural facts, which helps when estimating how long a "simple chat feature" will really take.
Design a RAG Pipeline
Retrieval-augmented generation built stage by stage: chunking documents, embedding them, searching by similarity, the recall and precision dial on top-k results, and where each stage breaks in production. It prepares you to ask sharp questions when answers look wrong, such as whether the problem is retrieval, chunking or the model, and to read a status update on search quality.
LLM Pricing Comparator
Enter tokens per call and monthly volume, and the comparator ranks models by cost, showing that input and output tokens are priced separately and that model choice dwarfs most other levers. Try your own numbers for a feature you are scoping. You will leave able to say what a plan costs per month, not just that it is "affordable".
The AI Cost Engineering Handbook
Explains where an inference bill comes from: per-token pricing, why output tokens usually dominate, and how caching, routing and shorter prompts cut spend. It includes an editable cost calculator and a list of common pitfalls. Use it to prepare for the budget conversation and to tell a real saving from one that quietly hurts quality.
Context Budget & Cost Planner
Model the system prompt, tool definitions, conversation history and retrieved passages for a feature, and see tokens and cost per call, then at 1,000 and 1,000,000 requests. The point is to feel how a long history or a large retrieval payload multiplies cost and latency. Run your own feature through it before you commit to a pricing plan.
The Agent Evaluations Handbook
Answers the question every PM eventually asks: how do we know the AI is good? It introduces tasks, agents and graders, the three families of grading (code checks, LLM judges, human review) and why a single successful run proves little. You will be able to ask for an eval plan and judge whether a launch threshold is meaningful.
The Guardrails Engineering Handbook
Guardrails are the input and output filters around a model, and this handbook shows both their value and their price: every layer catches more bad output but also blocks more good requests. The defense-in-depth math helps you reason about stacking filters. It prepares you to set an acceptable false-positive rate for your product, not just ask for "more safety".
The AI Security Handbook
Prompt injection is presented as a design problem, not a filter problem, along with the combination of private data, untrusted input and outbound action that makes agents risky. Data leakage, supply-chain risk and security evals in CI follow. Read it before a security review so you can ask what happens if the model is hijacked, and what it could reach.
The EU AI Act Handbook
A practical reading of the EU AI Act for people who ship products: regulation by risk tier, what makes a system high-risk, and the classification rule, with a short exercise for placing your own feature. It is not legal advice. It gives you enough shape to raise the right question with counsel early, before the roadmap commits you.
Design an AI Agent System
The closing step shows what happens when a feature grows into an agent: a goal, a plan-act-observe loop, typed tool calls, a safe executor and memory. Seeing the pieces explains why agents cost more, fail in stranger ways, and need the evals and guardrails you just studied. Use it to decide when an agent is warranted and when a fixed workflow is enough.

A study plan

Week 1
Read AI for Product Managers, The Prompting Handbook, then LLM vs SLM and RAG vs Fine-Tuning. Write a half-page note on a feature you own: which model size, whether it needs retrieval or tuning, and the reason for each choice.
Week 2
Skim the ChatGPT and RAG pipeline system designs for the shape of the system, not every detail. Then run your feature through the pricing comparator and the context cost planner, and note the monthly cost at three realistic traffic levels.
Week 3
Read AI Cost Engineering and The Agent Evaluations Handbook. Draft an eval plan for your feature in one page: what examples, who or what grades, and what score blocks a launch. Share it with an engineer and collect disagreements.
Week 4
Cover guardrails, AI security and the EU AI Act, then finish with the agent system design. Write a short risk list for your feature with an owner for each item, and decide whether it should stay a fixed workflow or become an agent.

Questions about this path

Do product managers need to learn to code to work on AI products?

No, but you do need to understand the moving parts well enough to make decisions. That means knowing what a prompt, a context window, retrieval and an evaluation each do, roughly what a request costs, and where the main risks sit. You can reach that without writing code by using interactive calculators and reading system designs. Coding experience still helps when you want to prototype, but the decisions in this path are about trade-offs, not syntax, and engineers will respect pointed questions more than a borrowed vocabulary.

How do I decide between RAG and fine-tuning as a PM?

Start by asking what is actually missing. If the model lacks facts that change or live in your documents, retrieval is usually the better fit because you update the data without retraining. If the facts are fine but the tone, format or task behavior is off, fine-tuning may help. Many products need both, and a better prompt solves more problems than people expect. Ask the team to show examples of failures first, since the type of failure usually points to the right fix.

How can a PM estimate the cost of an AI feature?

Estimate tokens per request for the system prompt, history, retrieved context and the answer, then multiply by the price per million input and output tokens and by monthly requests. Output tokens cost more than input, so long answers matter. Then test the sensitivity: double the history length, switch to a smaller model, add caching, and see what moves. The pricing comparator and the context cost planner in this path do this arithmetic, so you can check an engineer's figure rather than accept it.

What should a PM know about AI evaluation before launch?

You should be able to ask four things: what examples make up the test set and where they came from, how outputs are graded, what score blocks a release, and how quality will be watched after launch. A demo that works a few times is not evidence, because model output varies from run to run. Insist on a fixed set of realistic cases, a threshold agreed before the results arrive, and a plan for reviewing real traffic once the feature is live.

PMs shipping AI features need to reason about them without writing the code. This path builds that literacy — the AI stack, what is cheap versus genuinely hard, cost and evaluation, and the risk surface — so you can scope, prioritize, and pressure-test AI features with real understanding rather than vibes.

← All learning paths