Voice Agents Fail on the Same Three Call Types, and You Can Predict Which

The useful thing about voice agent failures is how unoriginal they are. Across deployments in different sectors, with different providers and different scripts, the calls that go wrong sort into three shapes. Not three topics, three shapes: the multi-intent call, the correction call, and the identity call. Every one of them is visible in call recordings you already hold, which means the failure profile of a system you have not bought yet is something you can estimate this week.

That matters because the alternative is finding out in production. A pilot runs on the calls somebody imagined, launch exposes it to the calls that arrive, and the gap between those two sets is almost entirely these three shapes.

Shape one: the call carrying more than one intent

A caller rings to move an appointment and, in the same breath, asks whether the earlier invoice was received and mentions their phone number has changed. One call, three requests, no pause between them.

The mechanism that breaks here is intent routing. A voice agent classifies the caller's utterance and follows a path. Given three intents in one utterance it typically latches onto whichever is strongest in the wording, handles that, and discards the rest. The caller hears their appointment moved, assumes all three items were dealt with because nobody said otherwise, and hangs up. The invoice question is never answered and the phone number is never updated. The call is logged as contained and successful.

That is the part worth dwelling on. Multi-intent failures are silent. They produce no angry caller and no transfer, so they never appear in a failure metric. They appear three weeks later as a missed appointment reminder sent to an old number.

Two things reduce it, and neither is a better model. First, an explicit sweep before closing: the agent restates what it handled and asks whether there was anything else, then treats the answer as a fresh classification rather than a courtesy. Second, logging the count of distinct requests detected per call, so a call that handled one of three shows up as a partial rather than a success.

Shape two: the correction

The agent mishears something, the caller corrects it, and the correction is the hard part. Not the mishearing.

"Tuesday the twelfth." "Right, Thursday the twelfth." "No, Tuesday." What the agent now receives is a fragment with no field attached. Human beings resolve this instantly because we hold the whole exchange as a shared object. A turn-based system has to work out which of the four values it has collected the word "Tuesday" is meant to overwrite, and cheap implementations either overwrite the most recent field regardless or restart the collection sequence from the beginning.

Restarting is the worse outcome. It makes callers give up, because their invested effort is visibly discarded and there is no reason to expect the second attempt to differ. Names and postcodes make it more acute, being exactly the values speech recognition gets wrong most often and exactly the values that must be right.

The design response is to hold collected values in addressable state, each carrying its own confidence, and to let a correction target one field rather than resetting the flow. Reading values back in a format that makes errors easy to catch helps too: spelling a surname, giving the day name alongside the date. That costs a few seconds and converts a silent error into a correction the caller can make on the spot.

Shape three: identity and authority

The caller wants something that requires knowing who they are, or requires permission the agent does not have. Cancel a policy. Discuss someone else's account. Approve a refund outside the normal window.

A voice on a phone line is not authentication. Caller ID is weak evidence, a date of birth is a shared secret with a long history of being shared, and no combination of spoken answers is sufficient basis for an action that moves money or changes a legal relationship. The failure mode is not occasional error. It is that a system designed to be helpful, under pressure from an insistent caller, will do something it should not, and a recording of it exists.

The authority half is the same shape without the security framing. A caller pushing for a discount will get one from an agent with no discount policy in front of it, because the model will produce a plausible number. No wording of a prompt reliably prevents this. The only durable answer is that the capability does not exist in the agent at all.

The three shapes side by side

Failure shape	What breaks	Why it is missed	The fix that works
Multi-intent	Secondary requests silently dropped	Logged as a successful contained call	Explicit sweep before close, plus per-call intent counts
Correction	Corrected value applied to the wrong field, or flow restarts	Shows as caller abandonment with no cause attached	Addressable slots with per-field confidence, read-backs on names and dates
Identity and authority	Action taken without sufficient basis	Rare, so it survives a small pilot	Remove the capability, route to a verified channel

The middle column matters most. Each failure hides in a different place: the first behind a good metric, the second behind an unexplained drop-off, the third behind low frequency. None are hard to fix once seen, and all are easy to ship without noticing.

How to predict your own mix before buying anything

This is the part you can do without a vendor in the room. Pull one hundred inbound calls from a normal week, spread across hours rather than taken from a single morning, and score each one against the three shapes:

1. Count the distinct requests in the call. Anything above one is a multi-intent candidate. The proportion above one is your multi-intent exposure, and in some sectors it is most of the call volume.
2. Mark every call where the person on your side had to correct or re-confirm a value. Note which values: names, dates, addresses, reference numbers. That list is your correction risk profile, and it tells you which fields need read-backs.
3. Mark every call that touched identity verification or required a decision only certain staff can make. That proportion is the share of your call volume that a voice agent should not attempt at all, which is the honest ceiling on containment.

Those three percentages are more informative than any vendor benchmark, because they describe your callers rather than someone else's. They also feed straight into a test suite: the awkward calls you found are the fixed input set to run against any candidate system, before and after every change. The method in our guide to testing an AI integration before it goes live applies here with one adjustment for voice, which is that the audio matters. Test with recordings and accents and background noise, not with typed transcripts, because transcription error is the input to two of the three shapes.

Note what this exercise is not. None of it is a data readiness problem in the machine learning sense, because a voice agent handling enquiries needs no training history at all. The distinction drawn in our guide to whether your data is ready for AI is the relevant one: what a voice deployment needs is access and permission, not volume. The agent must reach your calendar, your customer records and your current policies at the moment the phone rings.

What to do next

Score the hundred calls. It takes an afternoon and it produces three numbers that change how you scope the work, mainly by shrinking the ambition to something that will hold. Keep the difficult calls as your test set.

Then read those numbers against a realistic account of the envelope, which is what our guide to what an AI receptionist can and cannot do is for, and bring the scored set to any discussion about AI voice assistants for inbound calls. A vendor conversation held against your own difficult calls goes very differently from one held against a demo.