Most teams design their AI agent backwards. They spend weeks tuning answers and almost no time deciding what the agent should refuse to answer. Then a customer asks about a refund on a damaged order, the agent confidently quotes the wrong policy, and the team quietly turns the whole thing off.
The quality of an AI support agent is not decided by how much it can answer. It is decided by how well it knows its own edges.
Handoff is a design decision, not a failure
There is a habit of treating escalation as the agent losing. It is the opposite. An agent that hands off cleanly at the right moment is doing the most valuable thing it can do: protecting the customer from a confident wrong answer, and protecting your team from having to undo one.
Set the target accordingly. You are not aiming for a 100% deflection rate. You are aiming for a high resolution rate on the questions the agent should own, and a fast, well-briefed handoff on everything else.
Three signals that should trigger a handoff
In practice, almost every escalation worth making comes down to one of three signals.
Low confidence in retrieval. If the agent could not find a passage in your knowledge base that actually answers the question, it should not compose an answer from adjacent material. Retrieval quality, not model quality, is the thing to watch — the model will happily write a fluent paragraph from three loosely related documents. Set a floor, and hand off below it.
High-consequence intent. Some questions are risky to get wrong regardless of confidence: refunds, cancellations, billing disputes, data deletion, anything with a legal or financial consequence, anything touching a promise about delivery dates. Route these by intent, not by confidence. A very confident wrong answer about a refund is worse than an unconfident one.
Customer state. Frustration, repetition, and explicit requests for a person are all handoff signals. If a customer has asked the same thing twice, the agent has already failed once. Trying a third time makes it worse. Watch for repeated messages in a short window, all-caps, and phrases that ask for a human directly.
A useful default: the agent owns informational and transactional-lookup questions, and hands off anything that changes money, changes an order, or changes an account.
What you pass along matters more than how fast you pass it
The most common handoff failure is not being too slow. It is arriving empty-handed. The customer explains their problem, waits, then gets a human who asks them to explain it again. That single moment undoes any goodwill the speed bought you.
Whatever your stack, the handoff payload should carry:
- A one-line summary of what the customer wants
- What the agent already tried, and what it told them
- The customer’s order, account or ticket identifier if it was resolved
- Why it escalated — which of the three signals fired
- The full transcript, collapsed but available
The agent should also tell the customer what is happening, in plain language, with an honest expectation: someone from the team is picking this up, and roughly when. Silence during a transfer reads as being dropped.
Instrument the seam, not just the agent
Teams usually track deflection rate and stop there. Deflection alone will make your agent worse — it rewards answering things it should have escalated. Track the seam instead.
Handoff rate by intent. Not a single number. If refunds escalate 90% of the time, that is correct. If your top FAQ escalates 40% of the time, your knowledge base has a gap.
Resolution after handoff. How often does the human resolve it on first reply? If that number is low, your handoff payload is too thin and agents are re-diagnosing from scratch.
Repeat contact within 48 hours. The honest measure of whether anything was actually resolved. A conversation the AI closed that reopens two days later was never deflected — it was deferred, and it cost you twice.
Escalations the agent should have made. Sample resolved conversations weekly and count the ones where the agent should have handed off and didn’t. This is the number that catches confident wrongness, and nothing else will.
The mistakes that cost the most
Tuning for deflection. The moment deflection becomes the headline metric, every incentive points toward answering things the agent shouldn’t. Pair it with repeat-contact rate or don’t report it at all.
One global confidence threshold. Confidence means different things for “what are your opening hours” and “why was I charged twice.” Threshold by intent category.
Hiding the handoff. Customers can tell. Making the transition explicit — and making the human’s first message reference what the agent already learned — turns an escalation into evidence that the system works.
Never revisiting the boundary. The set of questions your agent should own grows as your knowledge base does. Review escalation reasons monthly; the top recurring reason is usually a missing article, not a model limitation.
Where to start
Pull your last 200 conversations. Sort them into three buckets: the agent should own this, a human should own this, and it depends. The third bucket is where your real design work lives — for each one, write down what would have to be true for the agent to handle it safely.
That list becomes your knowledge base backlog and your escalation rules at the same time. It is a better first week than any amount of prompt tuning.
