Vibe Engines is free, interactive AI engineering & system design — runs in your browser. Pick a learning path

Forward Deployed Engineer / Proving value & ROI

FDE Track~15 min readUpdated Jul 2026

Practitioner

Proving it worked,
before renewal

MIT’s NANDA study found roughly 95% of enterprise AI pilots produced no measurable profit-and-loss impact. Read that carefully: no measurable. A large share of those systems did something useful and could not demonstrate it, because nobody instrumented anything until the budget conversation. This is how not to be in that 95%.

The gap that kills deployments

The uncomfortable asymmetry

“It works” is your opinion. “Adjudication time went from 41 minutes to 9, on 2,300 cases, over eight weeks” is a fact your champion can take into a meeting you are not in.

At renewal, your champion has to defend this spend against three other things competing for the same budget, in a room where you are not present. Everything you build toward this is really about arming that one conversation.

Two things make this specifically an FDE problem rather than a product-analytics problem. First, you are usually measuring a workflow change inside someone else’s organisation, where you do not own the systems the baseline lives in. Second, the measurement has to be agreed before the result is known — a metric proposed after the fact is always suspected of being chosen to flatter.

The commercial logic behind the whole role rests on this. a16z’s services-led-growth thesis is that companies deliberately accept lower-margin deployment work because it produces defensible customer outcomes.⁸ That bargain only holds if outcomes are demonstrable. An FDE who cannot prove impact is a cost centre wearing an engineer’s badge.

Instrument on day one, not at renewal

The single highest-return hour of an engagement is capturing the baseline before you change anything.

Week 1

Capture the baseline while it still exists

Time the manual workflow with a stopwatch if that is what it takes. Count the cases. Count the errors. Get last quarter’s numbers from whoever has them. Once your system is live the baseline is gone forever, and reconstructing it after the fact is both painful and unconvincing.

Write it down and send it to your champion the same week: “here is what today looks like, agreed?” That email is your most valuable artefact at renewal.

Week 1

Agree the one number

Not a dashboard of twelve. One primary metric that everyone signs off on, with two supporting ones. If the room cannot agree on one, you have discovered a scoping problem, not a measurement problem — and better now than in month four.

Always

Build the telemetry into the thing you ship

Who used it, when, on what, what they did with the output, and whether they overrode it. Adding this later is an awkward retrofit; adding it in week two costs almost nothing and is the difference between evidence and anecdote.

→ Override rate: the most under-used metric in enterprise AI

Log every time a human changes the system’s output. It is simultaneously a quality signal, a trust signal, and a free stream of labels for your golden set. A falling override rate over eight weeks is the most persuasive chart you can put in a business review, because it shows the humans deciding to trust it.

The metrics that survive scrutiny

Four tiers. A business review needs at least one from each of the first three.

Tier	Metric	What it proves	How it is attacked
Adoption	Weekly active users of the workflow (not the app)	People chose to use it without being told to.	“They were mandated to.” Counter with voluntary-use segments.
Adoption	Share of eligible cases going through the system	It handles real volume, not a demo slice.	“Only the easy ones.” Counter by segmenting by difficulty.
Workflow	Time per case, before and after	The work actually got faster.	“The baseline was inflated.” Counter with the week-one signed-off email.
Workflow	Deflection rate — cases resolved without a human	Capacity was genuinely created.	“Quality dropped.” Counter with eval results on the same period.
Quality	Eval score on the frozen golden set, over time	Speed did not come from cutting corners.	“Your test set is easy.” Counter with stratification and the human baseline.
Quality	Override rate, trending	Trust is increasing among the actual users.	Hard to attack. This is why it is the strongest chart you have.
Business	Hours returned × loaded cost, or revenue/loss moved	Money.	“Those hours were not redeployed.” See the next section.

→ Never report a vanity metric

API calls, tokens processed, documents ingested, “queries served”. These measure your system’s activity, not the customer’s outcome, and a sophisticated buyer knows the difference. Including them dilutes the metrics that do matter and signals that you could not find a real one.

Making an ROI claim a CFO will accept

Most engineer-authored ROI numbers fall over on the same three objections. Pre-empt all three and the number survives.

# the arithmetic is trivial. the defensibility is not.
hours_saved = cases_processed × (minutes_before − minutes_after) / 60
gross_value = hours_saved × fully_loaded_hourly_cost
net_value = gross_value − licence − infra − their_internal_effort

# then subtract the three objections below, in writing, before they are raised.

Objection 1

“Saved hours are not saved money”

Correct, and it is the objection that sinks most AI business cases. Nobody was made redundant, so the salary line did not move. The fix is to say what the hours became: a backlog cleared, a queue that no longer needs contractors at quarter-end, a headcount request withdrawn, cases handled without the seasonal temp hires. Get your champion to name the redeployment before you write the number.

Objection 2

“How much of that was you?”

Other things changed in the same period. Never claim the whole delta. Segment: compare teams that adopted against teams that did not, or compare case types the system covers against ones it does not. A defensible partial attribution beats an indefensible total one, and offering the caveat yourself buys enormous credibility.

Objection 3

“Will it hold?”

Eight weeks of improvement can be a novelty effect. Answer with the trend line, the override rate falling rather than flat, and the drift monitoring that will catch degradation after handoff. This is where your eval work pays a commercial dividend.

→ The credibility move

Report a range with the assumptions listed, and include the case that did not work. A single confident number reads as marketing; a range with stated assumptions and one honest failure reads as engineering — and engineering is what they are buying from you.

The business review

Forty-five minutes, quarterly, and it decides the renewal. Build it once and reuse the structure.

Minutes	Section	Content
0–3	The answer	The one number and its direction. No build-up. If they leave after three minutes they should have the point.
3–10	Adoption	Who uses it, how often, share of eligible volume. Segment by team — the variation is where the next quarter’s work is.
10–18	Quality	Eval trend on the frozen set, override rate, incidents. Include a regression if there was one, with what you did.
18–28	Value	The ROI range, assumptions listed, attribution caveated, redeployment named.
28–38	What is not working	The workflow with low adoption and why. Do not skip this. A review with no problems reads as a review that was not looking.
38–45	Next	Three options with effort and expected impact. Let them choose — ownership of the choice is what makes the next quarter funded.

→ Who should present it

Where possible, your champion — with you in the room to answer detail. A win presented by the customer’s own person is worth several times the same slides presented by the vendor, because internally it is their success, and people fund their own successes.

The evidence layer

Half of this review is your eval work

The quality section is the frozen golden set over time. If that does not exist, the review has an opinion where it needs a chart.

Evals for client work Eval Builder Lab Handoff & enablement Incidents & SLAs

The economics that pay your salary

Worth understanding, because it explains which deployments get resourced and which quietly stop.

An FDE is expensive. The reason companies employ them anyway is a straightforward margin argument, and it is the argument a16z made explicitly: trade gross margin for deployment depth, because depth is defensible.⁸ Enterprise deals that require heavy deployment work carry contract values large enough to absorb the cost of a person, and the resulting integration is hard for a competitor to displace.

What this means for your work

Your value is measured against the contract, not against tickets closed.
Anything you build that generalises into the product improves the margin permanently — which is why the generalisation memo is the rung that moves careers.
Protecting a renewal is worth more than adding a feature.
Reducing the hours the next deployment needs is worth more still.

Signals your deployment is economically unhealthy

Nothing you build ever lands in the product.
Your time is measured as utilisation rather than as outcomes.
The customer’s spend is flat while your hours rise.
No one can name the metric this deployment is supposed to move.

The last bullet is the one to act on immediately. A deployment nobody can attach a number to is a deployment that will be cut in the next budget cycle regardless of how good the engineering was — and it is why this handbook belongs to the practitioner track rather than the commercial one. See also the services-trap checklist.

The picture

The picture: from baseline to renewal

The top row happens before anyone knows whether the system works, which is what makes the bottom row believable. Each box feeds the next: the signed baseline is the “before”, the agreed metric says what “after” means, telemetry collects it without anyone remembering to, and eight weeks of it becomes the evidence the review is built on. Skip the first box and every later box has nothing to compare against.

By the numbers

Worked example: turning 41 minutes to 9 into a defensible number

The page’s headline result: adjudication fell from 41 minutes to 9 on 2,300 cases over eight weeks. Everything else below is illustrative: a comparison team, a $70 fully loaded hourly rate, and the period’s costs. Follow the arithmetic, then swap in your customer’s figures.

Step	Question	Arithmetic	Answer
1	Minutes saved per case	41 − 9	32 min
2	What changed anyway? A team without the system went 41 → 37	41 − 37	4 min that are not yours
3	Attributable minutes per case	32 − 4	28 min
4	Hours returned	2,300 × 28 ÷ 60	1,073 h
5	Hours the champion can name a use for	75% (backlog cleared); 50% as the low case	805 h (low: 537 h)
6	Gross value	× $70 loaded rate	$56,350 (low: $37,567)
7	Costs for the period	$12,000 licence + infra, + 60 h of their staff × $70	$16,200
8	Net value, eight weeks	gross − costs	$40,150 (low: $21,367)
9	The naive claim, for contrast	2,300 × 32 ÷ 60 × $70, no costs	$85,867

Report “$21k to $40k net over eight weeks”, list the assumptions in steps 2, 5 and 7, and the number survives the room. The naive $85,867 is more than twice the top of that range. The first CFO to ask “and how much of that was you?” or “where did the hours go?” will knock it down, and your credibility goes with it. Do not annualise yet. Multiplying eight weeks by 6.5 assumes the volume and the gain both hold, which is the third objection, and the trend line is what answers it.

Trade-offs

Cost and trade-offs of measuring well

Here Time is the effort it takes you. Space is what you get back for it at renewal.

Case	Time	Space	Why
Stopwatch baseline in week 1	A few hours of watching and counting	A signed “before” that nobody can dispute later	Cheapest evidence you will ever collect, and the only time you can collect it.
Reconstructing the baseline at renewal	Days of archaeology in systems you do not own	A number everyone suspects	Memory of “how slow it used to be” inflates. The customer knows it.
Telemetry built in during week 2	A small part of the build	Adoption, override and time-per-case data for free, every week	Retrofitting it later means changing a live workflow the users already trust.
A comparison group or segment	Finding a team or case type the system does not cover	An attribution you can defend	Without it, every seasonal or process change in the period counts as yours, until someone asks.
One metric, not twelve	A harder scoping conversation up front	One number your champion can repeat	Twelve metrics guarantee one moves the wrong way, and that becomes the meeting.
A range, not a point	Listing your assumptions	Survives the CFO’s first question	A point estimate is only as strong as its weakest assumption.

Watch for

Common mistakes

Reporting the mean time per case
Case times are skewed: a few three-hour cases drag the average around. A change in the mix of hard cases can move the mean more than your system did. Report the median and the 90th percentile, before and after, on the same case types.
Letting the metric’s definition drift
Halfway through, “resolved” starts to include “escalated to tier 2”, and deflection jumps. Write the definition down with the baseline, and if it has to change, recompute the baseline under the new one.
Counting the same minutes twice
A deflected case saves its whole handling time. Count it as deflected and include it in the faster time-per-case average, and the savings are double-counted. Split the cases into deflected and assisted, then value each group once.
Using an hourly rate finance does not recognise
You picked a loaded rate; finance uses another. The whole number is now an argument about the rate. Ask finance for their figure in week 1 and cite it.
Comparing across seasons
A baseline taken in the quiet month against a live period at quarter-end makes the system look worse; the reverse flatters it. Match the periods or use the comparison group, which lives through the same season.

In the wild

Where this shows up in the real world and in interviews

Pilot to production
The same baseline, metric and range become the business case for moving from a pilot to a funded rollout. Procurement and finance read it, often without you there to explain it.
Difference-in-differences
Step 2 of the example is a standard method from economics: compare the change in the treated group with the change in an untreated group over the same period. Naming it helps with analytically minded buyers.
Expansion and references
A measured win in one department is how the next department gets funded, and, with the customer’s permission, how a reference call goes well. Unmeasured wins do not travel.
The FDE interview
“Tell me about the impact of something you deployed” is a standard question. Answer with the baseline, the agreed metric, the number as a range, what you could not attribute, and one thing that did not work. That shape is the evidence they are hiring for.

Self-check

Check yourself

Usage keeps rising, but the override rate has sat flat at 30% for eight weeks. What does that tell you, and what do you do?

People are using it, perhaps because they were told to, but they don’t trust it more over time. Find out what the 30% of overrides have in common (case type, team, field). Those are labels for your golden set and the next quarter’s fix. Don’t present flat overrides as a trust win.

An adopting team’s time per case went 40 → 10 minutes. A similar team without the system went 40 → 30 in the same weeks. How many minutes per case can you claim?

20. The adopting team improved by 30, but 10 of that happened without the system too. 30 − 10 = 20. Claiming 30 invites the attribution objection; offering 20 yourself earns credibility.

Your champion says the team saved 1,200 hours. Nobody left, no backlog shrank, and no hires were cancelled. What goes in the review?

Not 1,200 hours of money. With no named redeployment, the “saved hours are not saved money” objection wins. Report the time-per-case and quality results, and work with the champion to find where the hours actually went before you put a dollar figure on them.

A stakeholder wants twelve metrics on the dashboard “to be thorough”. How do you respond?

Agree one primary metric and two supporting ones, and keep the rest as diagnostics. If the room can’t pick one, you’ve found a scoping disagreement worth settling now. With twelve headline numbers, one will move the wrong way and take over the review.

Frequently asked

Quick answers

How do you prove the ROI of an AI deployment?

Capture the baseline in week one before anything changes, agree one primary metric with the customer up front, instrument adoption and override rate inside the product, then report hours or errors moved with the assumptions listed and attribution caveated. Pre-empt the three standard objections: that saved hours are not saved money, that other things changed too, and that the improvement may not hold.

What metrics should an FDE track for a deployment?

At least one from each of three tiers. Adoption: weekly active users of the workflow and share of eligible cases going through the system. Workflow: time per case before and after, and deflection rate. Quality: eval score on a frozen golden set over time, and override rate trending. Business value sits on top of those. Avoid vanity metrics like API calls or documents processed.

Why do most enterprise AI pilots fail to show impact?

MIT’s NANDA study found roughly 95% produced no measurable profit-and-loss impact, and the operative word is measurable. Many worked but could not be demonstrated, because no baseline was captured before the change, no metric was agreed in advance, and adoption was never instrumented — leaving only anecdotes at the budget conversation.

What is the override rate and why does it matter?

The share of system outputs a human changes before acting on them. It is simultaneously a quality signal, a trust signal, and a free source of labels for your golden set. A falling override rate over several weeks is the most persuasive chart in a business review, because it shows the users choosing to trust the system rather than being told to.

Who should present the quarterly business review?

Where possible the customer’s champion, with the forward deployed engineer in the room for detail. A result presented by the customer’s own person carries far more internal weight than the same slides from a vendor, because internally it becomes their success — and organisations fund their own successes.

Receipts

Sources

Every number on this page traces to one of these. Where a figure is self-reported or crowd-sourced rather than first-party, it is labelled inline.

Cited on this page

a16z — Services-Led Growth (Joe Schmidt, 4 Jun 2025) — the margin-for-moat thesis behind FDE hiring. a16z.com/services-led-growth/
levels.fyi — Palantir FDSE compensation — www.levels.fyi/companies/palantir/salaries/software-engineer/title/fdse
MIT NANDA “State of AI in Business” — 95% of pilots with no measurable P&L impact — report dated July 2025, available as a PDF; the figure here is cited via later press coverage (TechCrunch, 30 Jul 2026), which is what this link points to. techcrunch.com/2026/07/30/forward-deployed-engineers-are-the-ai-industrys-latest-talent-obsession/

Where this sits Customer-grade production Rung 9 of 11

What a demo has to survive to become a deployment: their cloud, their auditors, their identity provider, their on-call rota — and finally the handoff that ends your involvement.

Finished this one? 0 / 209 Handbooks done

NextThe Commercial Layer for FDEs Next on the “Customer-grade production” ladder · rung 10 of 11

Explore the topic

See this alongside everything else on the same subject — handbooks, system designs, challenges and tools, in one place.

Forward Deployed Engineering

More Handbooks

Explore more

Labs →

Paper Breakdowns →

The 95% Number, Examined