Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path
Scoping Question Log
A drill for the skill every discovery call actually tests. Two underspecified customer asks, twelve candidate questions each, and a budget of six — spend them, see what the customer really says, and get each question scored on whether the answer changes what you build. Ends with a copyable scoping doc, a coverage map across the six axes, and the thing that was really going on that you either found or missed.
“We want AI to handle our invoices.”
That constraint is the point. A real discovery call has room for about six questions that matter, and the skill being measured is which six.
Pick questions on the left. Each one costs a slot and returns what the customer actually says — which is the only thing that tells you whether the question was worth asking.
The scoring rule: 3 if the answer changes what you build, 2 if it changes the plan, 1 if it is nice to know, 0 if it is premature or has no wrong answer.
"We want AI to handle our invoices" is the start of a requirement, not a requirement, and the next six questions decide whether the project gets built or argued about. This drill puts you in two discovery calls with a budget of six questions from twelve candidates, shows what the customer actually says, and scores how much of the available signal you captured.
What it computes
Each brief has twelve candidate questions. Every question sits on one of six axes (Outcome, Data, Users, Constraints, Success, Risk) and carries a signal value from the scoring rule: 3 if the answer changes what you build, 2 if it changes the plan, 1 if it is nice to know, 0 if it is premature or has no wrong answer.
budget = 6 questions per session
score = sum of the values of the questions asked
max score = sum of the six highest values in the brief
signal % = score / max score × 100
coverage = distinct axes asked about, 0 to 6
Scopeable signal % ≥ 80 and coverage ≥ 4
Partial signal % ≥ 55 and coverage ≥ 3
Not scopeable yet anything elseThe panel shows the score as score/max and lights each covered axis. When the sixth slot is spent it adds the grade, the story that was really behind the ask, and a plain-text scoping doc: each question with the customer's answer under an axis tag, every untouched axis listed as open risk with its guiding question, and three next steps.
A worked example
The default brief is Kestrel Freight. Seven of its twelve questions are worth 3, so the best possible six score 6 × 3 = 18. Three sessions show how the rules separate them:
| Session | Questions and values | Score | Axes | Grade |
|---|---|---|---|---|
| Common opener | model choice 0, ERP API 1, feelings about AI 0, budget 1, invoice volume 3, today's process 3 | 8/18, 44% | 5 | Not scopeable yet |
| Mixed | today's process 3, invoice volume 3, where it can run 2, ERP API 1, budget 1, model choice 0 | 10/18, 56% | 4 | Partial |
| Strong | today's process 3, invoice volume 3, who resolves exceptions 3, where it can run 2, the success number 3, cost of a wrong approval 3 | 17/18, 94% | 6 | Scopeable |
The opener touches five axes and still fails, because 8 of 18 is under the 55% floor: breadth without signal does not count. The strong session gives up a point on the constraints question, worth 2, but covers all six axes, so its doc has no open-risk section. Six 3-point questions confined to Outcome, Data, Users and Success score a perfect 18 and also grade Scopeable, yet the doc lists Constraints and Risk as open, and the four-week ERP approval and the £500 and £5,000 overpayment thresholds stay undiscovered.
Assumptions and limits
- Values are authored judgments. Each 0 to 3 is the drill author's call for this brief, not a measurement; a question worth 1 here can be decisive in another engagement.
- Order does not matter. The score is a sum, so the best question asked last scores the same as asked first. In a real call an early answer changes which question comes next.
- Fixed answers. The customer says the same thing every time and never volunteers more. Real stakeholders digress, contradict each other and need follow-ups.
- Coverage is a count. One question on an axis marks it covered, however shallow.
- Two briefs. Halcyon Health, the second, has no 3-point Outcome question, so covering all six axes caps a session there at 17.
How to use the result
- Before each question, ask whether either answer would change what you build. If not, it is a 0 or a 1 and does not deserve one of six slots.
- Treat the open-risk list as your next agenda. Every untouched axis is a question to schedule, and it is usually Data or Risk.
- Convert every answer into scope. Measurable answers become acceptance criteria, such as 40 of the 60 exceptions resolved without a human; dependencies become dated assumptions.
- Ask the right person. Several low scores are reasonable questions put to the wrong stakeholder, such as an ERP API question asked of the VP of Finance.
How it works
- Two realistic underspecified asks, twelve candidate questions each.
- A budget of six: the constraint is the exercise.
- Each question scored 0–3 on whether its answer changes the build.
- Graded against the best possible six, with coverage across six axes.
- Exports a scoping doc with what you learned and the open risks.
Frequently asked questions
What makes a good clarifying question in a discovery call?
One whose answer changes what you build. Asking what happens today, step by step, routinely reveals that the stated problem is already solved and the real work is somewhere else. Questions with no wrong answer, such as how the customer feels about AI, cost a slot and return nothing you can design against. The test before asking anything is simple: if the answer came back either way, would I build something different? If not, the question can wait.
What are the axes a scoping session needs to cover?
Six: outcome (what changes in the business if this works), data (what exists, who owns it, whether you can actually get it), users (who does this work today and what they do instead), constraints (where it can run, what it can touch, who must approve), success (what number moves, measured how, agreed by whom) and risk (the cost of being wrong and how often that is acceptable). An axis you never asked about is where the surprise comes from, and it is usually data or risk.
Why limit the number of questions to six?
Because a real discovery call has room for about six questions that matter, and the skill being assessed, in the job and in the forward deployed engineer decomposition interview round, is choosing which six. Removing the budget removes the exercise: with unlimited questions everyone eventually covers everything, and nothing is learned about prioritization. The budget also makes the cost of a polite non-answer visible, because every slot spent on one is a slot not spent on data or risk.
How is the score in the scoping drill calculated?
Every candidate question carries a value from 0 to 3: 3 if its answer changes what you build, 2 if it changes the plan, 1 if it is nice to know, and 0 if it is premature. Your score is the sum for the six you ask, compared with the best possible six, which is 18 in both briefs. Signal at or above 80 percent with at least four axes covered grades Scopeable, 55 percent with three axes grades Partial, and anything less is Not scopeable yet.
What do you do with the answers?
Turn each one into either an acceptance criterion or an assumption with a date. Forty of the sixty exceptions resolved without a human is a testable acceptance criterion; the customer granting ERP access by week one is an assumption, and it only protects you if the statement of work also says delays shift milestones day for day. Anything still unknown when scoping ends is a scope argument already scheduled for month four.
What questions should I ask before building an AI project?
Start with the current process, step by step, because it often shows that the stated problem is already solved and the real work is elsewhere. Then ask for real examples of the data, who does the work today and how they decide, where the system may run and who approves access, the number that defines success and who agrees it, and the cost of a wrong answer. Leave model choice, search technique and budget for later or for other people: those answers rarely change the scope.