We built a Voice Agent That Handles 90% of our Calls Without a Human

Bezit built Jessica, a voice agent on Twilio Voice, Claude Haiku 4.5, and ElevenLabs, that identifies homeowners, handles interruptions, and resolves property tax questions without a human — cutting resolution time from 2 days to 1 hour.

Bezit is a property tax company — we file protests on behalf of homeowners and get their appraisal value reduced by the county office. That's the whole business. But underneath every protest is a mountain of phone calls: fee questions, status checks, document requests. Until recently, every single one of those calls had to land on a real person from our ops team.

I built Bezit IVR to change that. It's a voice agent — we call her Jessica in production — that answers the phone, identifies who's calling, pulls their case from production, and holds an actual conversation instead of a menu tree.

I put together a short walkthrough of it for Twilio's AI Startup Searchlight 2026 (Breakthrough Builders track), and I wanted to write up the same story here with a bit more detail than fits in a video.

Demo: https://www.youtube.com/watch?v=SGFO_i4FMp4

Pitch deck: https://drive.google.com/file/d/1L6ltX6c8iuV8PZjDPMKaCQbJ7ycAQj2B/view?usp=sharing

Not a reply box

The bar I set for myself wasn't "can it answer a question." Most IVR systems are reply boxes — you say a keyword, it maps to an intent, it plays back a canned response. That breaks the moment a caller says something unexpected, interrupts, or asks a follow-up. Property tax callers do all three constantly.

So Jessica needed to do five things a person on my ops team already does:

  • Answer — a non-customer can call in and ask general questions, no account needed.
  • Identify — an existing customer gets matched against production data by name, phone number, email, or property address.
  • Listen through interruptions — barge-in works both ways. If you cut her off mid-sentence, she stops, takes in what you just said, and responds to that, instead of finishing her original line first.
  • Follow up — tell her you're busy, and she calls back at whatever time you give her.
  • Escalate — anything legal or serious gets kicked to a human instead of Jessica trying to wing it.

Four layers

Here's how it actually works under the hood — four layers: entry channel, intelligence layer, action, and response.

flowchart TD
    A[Caller] --> B["Entry Channel<br/>Twilio Voice (inbound + outbound)"]
    B --> C["Intelligence Layer<br/>ElevenLabs STT → Claude Haiku 4.5 → ElevenLabs TTS"]
    C --> D{General or<br/>property-specific?}
    D -->|General question| E[Answer directly]
    D -->|Property-specific| F["Action Layer<br/>API call into production<br/>(tax bill, protest status, docs)"]
    F --> G["Response Layer<br/>Spoken answer back to caller"]
    E --> G

Twilio Voice is the entry channel — it receives inbound calls and is also how Jessica places outbound calls. The intelligence layer takes that audio, transcribes it with ElevenLabs speech-to-text, feeds it to Claude Haiku 4.5 to reason over, and turns the reply back into speech with ElevenLabs text-to-speech.

The brain decides what to do and when. If the question is general, it answers straight away from what it already knows. If it's property-specific, it triggers a function call into production — tax bill, protest status, address details — and answers from that, not a guess.

I went with my own real-time pipeline instead of a managed conversational-voice platform on purpose. A managed layer would've been the faster path — plug in a prompt, get a number, done — but barge-in and turn-taking are the two things that make a voice agent feel human or feel broken, and managed platforms give you limited control over both. Running it myself on top of Twilio Voice cost more time upfront, but it's the reason Jessica can be interrupted mid-sentence and actually pick up the thread instead of talking over the caller.

More than a voice agent — it's a portal

This isn't just a voice agent, it's a complete portal. I built it so I can see every call coming in and every call going out in one place, which is what lets my ops team manage everything without digging through email threads and call logs. Right now, inbound and one outbound flow — document follow-up — are live in production. Service requests go through one pipeline, general questions go through another.

If you want the business case and roadmap behind this (where it's going next, how the numbers below translate into ops time saved), that's what I put together in the pitch deck — I'll drop the link here once it's ready to share.

Two use cases from the video

I walked through both of these live in the demo — here's the gist of each.

No back-and-forth over email, no expiring link creating false urgency — just a text with what he needs, sent while he's already on the phone. For every call, inbound or outbound, I get a summary of what happened, any documents received, and call performance, all sitting in the portal afterward.

The numbers that matter to me

Before the voice agent, resolving a homeowner's request — a document, a status check, a fee question — took anywhere from zero to two days, and it usually meant bouncing between email, phone, and whatever channel the homeowner picked. Now that same resolution happens in zero to one hour, because Jessica calls, gets the answer or the document, and closes the loop in a single interaction.

The metric I care about most: resolved without a person went from 10% to 90%. Nine out of ten calls that used to need someone on my ops team now don't. That's not a marginal automation win for a team this size — it's the difference between ops spending their day on phone triage versus spending it on the cases that actually need judgment.

What's next

Right now inbound and the document-follow-up outbound flow are live. The portal already tracks every call's summary and documents, so the next flow is more of the same pattern: spot the trigger — a status change, a missed callback window, an escalation — and let Jessica make the call before a person even has to think about it.

If you want the full walkthrough with the live demo audio, it's in the video. And if you want the numbers, roadmap, and business case laid out properly, that's what the pitch deck is for.

SEARCHLIGHT_DECK.pdf