The big idea
Knowing when NOT to answer matters more than answering
A support agent that confidently answers every ticket, including ones where it genuinely doesn't have the right information, is worse than one that admits uncertainty and hands off — a wrong answer about someone's actual order or account erodes trust far more than a quick, honest escalation.
We'll build around that principle: grounding in real account state, confidence-aware escalation, tiered resolution by risk, seamless handoff, and measuring success by whether the issue actually STAYED resolved.
How to read this: Each step opens with a real design decision — make the call before I show you what ships. Watch the diagram grow, hover the boxes, and at the end cut account data access and try to social-engineer a refund to see the safeguards hold. Hit Begin.
Step 1 · The baseline
Answer from general knowledge alone
Simplest version: the agent answers every ticket using general product knowledge — documentation, FAQs, common patterns. What breaks when the real answer depends on something specific to THIS customer?
Design decision: The agent answers from general product knowledge alone. What breaks for account-specific questions?
The call: For questions that depend on this specific customer's actual account or order state, the agent has no way to know the real answer — it can only guess or produce a generic, potentially wrong response, which is worse than admitting it doesn't have the information. — General knowledge is fundamentally the wrong source for questions like "where is MY order" or "why was I charged this" — these require actual, current, customer-specific data, and answering from general knowledge alone means either a vague non-answer or a confident guess that could easily be factually wrong.
General product knowledge can't answer questions that depend on a specific customer's actual account or order state — attempting to anyway produces vague non-answers or confident guesses that may be factually wrong, which is worse than honestly not knowing.
Know the difference between general and account-specific knowledge: A support agent needs to recognize which category a question falls into — general knowledge it genuinely has, versus a fact that only exists in this customer's actual account data — and never blur the two.
Step 2 · Ground in real account data
Retrieve the actual facts, not just documentation
Fix Step 1: connect the agent to the customer's real account and order state.
Design decision: The agent needs to answer account-specific questions. What should it retrieve to do so accurately?
The call: The customer's actual, current account and order state, retrieved live from the systems of record — so the agent's answer is grounded in real, verifiable facts about this specific customer, not general patterns. — Retrieving the real, live account/order data means the agent's answer is actually correct for THIS customer, right now — not a plausible-sounding guess based on what's typical, but the genuine current state of their account.
Ground responses in the customer's actual, current Account/Order State, retrieved live from the real systems of record — so answers reflect genuine, verifiable facts about this specific customer, not general patterns or stale snapshots.
Ground generation in the right data: This is the same grounding principle behind RAG-based factual accuracy elsewhere, applied to support: the model's output is only as trustworthy as the data it's actually anchored to, and for support that data has to be THIS customer's real, live account state.
Step 3 · Recognize uncertainty
Confidence has to be a first-class signal
Even with real account data, some tickets are genuinely ambiguous, ii require judgment, or touch high-stakes situations (a billing dispute, an angry customer, a legal threat). Should the agent always attempt a full resolution?
Design decision: A ticket is ambiguous or high-stakes. Should the agent always attempt to fully resolve it itself?
The call: No — the agent should estimate its own confidence, and for ambiguous, high-stakes, or genuinely uncertain tickets, escalate directly to a human rather than attempting a guess. — Recognizing low confidence and escalating BEFORE attempting a risky guess avoids ever producing (and potentially acting on) a wrong response for exactly the tickets where being wrong matters most — confidence has to gate action, not just be assessed after the fact.
Build Triage + Confidence assessment as a first-class step: the agent estimates how confident it genuinely is, and low-confidence, ambiguous, or high-stakes tickets escalate before any resolution attempt — rather than guessing first and hoping it's right.
Confidence should gate action, not just be reported after: An agent that assesses confidence only retrospectively ("I was 60% sure") still already took the action. Real safety requires confidence to determine WHETHER to act autonomously at all, before any response is generated for the customer.
Step 4 · Tiered resolution by category and risk
Not every ticket needs the same treatment
Some tickets are simple and low-risk (a password reset, an order-status lookup); others are complex or consequential (a refund dispute, an account closure). Should every ticket go through the same resolution path?
Design decision: Ticket complexity and risk vary enormously. Should every ticket follow the same resolution path?
The call: No — tier resolution by category and risk: fully auto-resolve well-understood, low-risk categories; partially automate others (draft a response for human review); escalate the rest entirely to a human. — Tiering matches the level of automation to the actual risk and complexity of the category — trivial, well-understood issues get fast full automation, moderately complex ones get an assisted draft a human reviews, and genuinely complex or high-stakes issues go straight to a human, unautomated.
Tier resolution by category and risk: fully auto-resolve well-understood, low-risk tickets; partially automate (draft for human review) moderately complex ones; fully escalate genuinely ambiguous or high-stakes tickets — matching the level of automation to actual risk rather than one uniform process for everything.
Automation level should track actual risk, not be uniform: The same "scope autonomy to consequence" principle from other agent designs, applied here as an explicit three-tier system spanning full automation, assisted drafting, and full human handling.
Step 5 · Seamless handoff
The customer should never have to repeat themselves
A ticket escalates to a human agent. Should the human start from scratch, or receive what the AI agent already gathered?
Design decision: A ticket escalates to a human. Should the human start the conversation over from scratch?
The call: No — transfer the full conversation history and any partial diagnosis to the human agent via Handoff Context, so the customer never has to repeat what they've already explained. — A complete context transfer lets the human agent pick up exactly where the AI left off — reading what's already been established rather than making the customer re-explain their issue from the beginning, which is exactly the frustration escalation should avoid causing.
Transfer the full conversation and any partial diagnosis via Handoff Context to the receiving human agent — so escalation feels like a continuation, not a restart, and the customer is never asked to repeat what they've already explained.
Escalation shouldn't erase progress already made: The AI agent's work up to the point of escalation — understanding the issue, gathering relevant context — has real value even if it couldn't fully resolve the ticket. Losing that at handoff wastes it and directly harms the customer experience.
Step 6 · Measure success by outcome, not activity
"A response was sent" isn't the goal
How should resolution quality actually be measured — by whether the agent sent a reply, or by something else?
Track outcomes via the Outcome Tracker: whether the customer actually stopped contacting support about the SAME issue within a reasonable window, not just whether a response was sent. A ticket "closed" by the agent that the customer reopens or re-contacts about days later wasn't actually resolved — it just looked resolved in the moment.
Measure the outcome that actually matters: "Did we respond" is an activity metric; "did the problem actually go away" is an outcome metric — and only the second one tells you whether the system is genuinely working, a distinction that matters for any support or resolution system, human or automated.
Step 7 · Tone and de-escalation
A frustrated customer needs a different response, not just a correct one
A technically correct response delivered in a flat, generic tone can make an already-frustrated customer feel worse, not better.
Detect frustration or anger signals in the conversation and adjust accordingly — a more empathetic tone, and, for genuinely escalated emotional situations, prioritizing faster handoff to a human agent even for issues that might otherwise be auto-resolvable. Being technically correct isn't sufficient when the delivery itself affects whether the interaction actually resolves the customer's underlying frustration.
Correctness and delivery are both part of the response: A support interaction's success depends on more than factual accuracy — HOW something is communicated matters, especially when a customer is already upset, and a system that ignores tone is solving only part of the actual problem.
Step 8 · The sharp edges
Social engineering and gaming attempts
Two adversarial patterns: a customer confidently claiming authority they may not actually have to push through an unauthorized consequential action, and a customer repeatedly rephrasing the same request hoping to get a more favorable automated answer by chance.
Require actual identity/authorization verification before any consequential action (a refund, an account change) — a confident claim asserted in a chat message is never treated as proof of authorization, no matter how it's phrased. For gaming attempts, detect repeated near-identical requests and route them to human review rather than letting persistence alone eventually produce a favorable automated outcome by chance.
Design for the unhappy path: Account data down → escalate what can't be verified, keep serving what can. Social engineering → verification is required, not assumed from assertion. Gaming attempts → repetition itself is a signal for review. A support agent that only works for honest, straightforward customers is a demo; one that resists manipulation is a product.
You did it
You just designed a support resolution agent.
- General product knowledge can't answer account-specific questions — a confident guess is worse than an honest escalation.
- Grounding in real, live account/order state is what makes answers actually correct for a specific customer, not just plausible-sounding.
- Confidence assessment has to gate action BEFORE a resolution attempt, not just be reported after the fact.
- Resolution is tiered by category and risk — full automation, assisted drafting, or full escalation — rather than one uniform process.
- Escalation transfers full context so customers never have to repeat what they've already explained.
- Resolution quality is measured by whether the issue actually stayed resolved, not just whether a response was sent.
- Tone-aware de-escalation matters — correctness alone doesn't address an already-frustrated customer's experience.
- Consequential actions require actual identity verification, never just a confidently-asserted claim, and repeated gaming attempts are their own signal for review.