Beyond IVR: How Video Agents Are Replacing Phone Trees

A caller with a simple problem presses 3, then 1, then 0, and says "representative" louder each time while a recorded voice apologizes for not understanding. The phone tree accepts only the answers it was built to expect, so she has no way to explain what actually went wrong.

Contact centers are replacing interactive voice response (IVR) menus with systems that listen to what callers say, and IVR replacement is arriving in two waves. The first swapped keypads for AI voice agents that interpret open-ended speech. The second adds a face, so the system can see confusion or worry as it happens and respond with the kind of attention people call presence.

What IVR does and where it breaks down

An IVR system answers a call and routes the caller through a scripted decision tree. Older systems accept only dual-tone multi-frequency (DTMF) keypad tones. Newer ones add speech recognition tuned to a short list of expected words, so callers still have to fit their problem into options someone wrote in advance.

Menus get deeper with every new product. The tree can't parse a sentence like "I was charged twice after I canceled." A caller whose problem has no matching branch ends up guessing which number comes closest.

In a Gartner self-service survey published in August 2024, 45% of customers who started in self-service said the company didn't understand what they were trying to do.

AI voice agents and the first wave of IVR replacement

Natural-language voice agents remove the numbered menu. A caller states the problem in her own words, and the agent works out the intent, asks a follow-up when it needs one, and then resolves the request or routes it with context attached.

Next to a keypad tree, voice agents change three things in daily operations.

• Routing by intent: The call goes where the caller's words point, so a billing dispute stops landing in technical support because someone pressed 2 out of habit.
• Fewer steps to an answer: Callers skip the menu recitation and reach a resolution or the right team sooner.
• Context on handoff: When a person takes over, the agent passes along the transcript and the caller's stated intent, so she doesn't have to repeat an account number.

Many teams move one line of business at a time, such as billing or appointment scheduling, and prove it works before moving the next.

How to replace IVR in stages

A staged plan keeps the risk low, because each stage has to prove itself before the next one begins.

1. Audit the call flows. Pull volume per path, abandonment at each menu level, transfer rate, and handle time.
2. Pick a first flow. Choose a bounded, high-volume flow where a bad call is costly but not catastrophic, and defer flows that depend on backend data the agent can't reach.
3. Design from real calls. Build intents from call transcripts instead of the old menu structure, since callers describe problems in their own words.
4. Plan the handoff. Decide how context moves to a person and how fast. In Gartner's 2026 customer survey of 3,566 customers, 87% said companies using generative AI for service must provide access to a human agent.
5. Connect the systems. Link telephony, CRM, and billing so the agent can act on a request instead of only answering questions.
6. Measure against the baseline. Compare containment, escalation rate, first-contact resolution, and satisfaction on escalated calls with the IVR's numbers for the same flow, and expand once they hold.

A shorter call only counts as an improvement when the caller's problem was resolved, so pair handle time with satisfaction and resolution.

The signals voice alone cannot read

Even a well-measured voice agent leaves the caller's face out of the conversation. A phone line carries words and tone, but not a wince, a glance away from the screen, or the tightening around the mouth that comes before someone says "I'm fine."

Cues like these can change the right answer in calls like intake and hardship requests. A patient describing chest tightness during intake may understate it out loud while her posture says otherwise. A borrower asking about hardship options can sound calm while close to hanging up from embarrassment.

Multimodal perception systems try to recover those cues by processing audio and video together and describing what they hear and see for a language model to reason over. The aim is to catch a mismatch between what a caller says and how she says it.

What a video PAL adds to a phone line

Interpreting a caller's face and voice together takes a system built for live video. Tavus is the human computing company, building PALs that see, hear, understand, and respond in real-time conversations.

A Personified Application Layer (PAL) is a real-time application a caller talks to face to face, one that sees, hears, remembers, and responds. Teams build PALs on the Conversational Video Interface (CVI), which runs four components as one closed loop.

• Timing: Sparrow-2 predicts who holds the conversational floor frame by frame, timing replies to the moment a human listener would speak. In preliminary results on TurnBench, it posts 92.4% end-of-turn recall and 97.4% interruption recall.
• Perception: Raven-1 fuses tone, expression, hesitation, and body language into one signal. It describes that signal in natural language for the LLM to reason over, with audio perception under 100ms and rolling context no more than 300ms stale.
• Reasoning: The large language model (LLM) layer reasons about what to say and do next.
• Rendering: Phoenix-4.5 renders the PAL's face, head, and upper body in real time, with audio-to-video latency under 130 milliseconds.

A PAL could handle a double-charge call to an internet provider differently. The customer says the credit "sounds fine" while he keeps looking down at his bill, and Raven-1 catches the mismatch. The LLM layer asks which line item still worries him, and Sparrow-2 holds the floor open while he finds the number, so the call resolves without a transfer.

Voice-only mode is the practical entry point for phone-heavy flows. Tavus offers it on CVI with the same perception and turn-taking layers and face rendering switched off, so a team can launch audio first, validate it, and add the face on the same stack.

Where video PALs are replacing phone trees

The strongest candidates are calls where a menu misses the caller's state, as in intake, financial conversations, and support.

Healthcare intake and follow-up

A clinic's phone menu sorts calls into scheduling, refills, and billing. A patient calling about a symptom after a procedure has to pick the closest option, and the menu can't tell how serious it is. When Raven-1 catches a wince alongside a pause in her answer, the LLM layer can slow the pace and check whether the pain needs a nurse now.

Careflick's companion PALs for seniors found that 97% of users chose video over voice-only when offered both. For healthcare deployments, HIPAA compliance is available on the Enterprise plan.

Sensitive conversations in financial services

A borrower who has fallen behind on a mortgage payment reaches a menu offering balance, payment, and payoff options, and none of them says hardship. She picks the closest one, and the real reason for the call comes out only after a transfer or two.

A PAL can open with a plain question about what is going on, and Raven-1 can flag when her calm words and tense expression disagree. It can explain which hardship programs exist, while individualized advice stays with licensed staff, so the call moves to a specialist as soon as she asks for one.

Inbound support and sales qualification

Menu selection is a weak proxy for intent. A customer whose invoice is wrong because a seat count didn't update may be transferred twice before reaching billing. A PAL that hears the problem and sees the customer squint at a shared screen can confirm the seat count before acting.

Sales qualification works the same way, because a buyer's first question rarely matches a menu option. Salesforce built Piper, an AI sales development rep (SDR), on Tavus to take those conversations in real time.

Start with the call where seeing the caller changes the answer

A customer says the credit sounds fine while his eyes stay on the line item he disputed. A system that notices that mismatch and slows down gives him presence, the sense that someone on the other end is paying attention to what he means.

Callers have always wanted the company on the other end to listen this closely. Tavus builds PALs for the intake and hardship conversations phone trees handle poorly, with a person always one handoff away.

See it for yourself. Book a demo.