# Choosing a Voice Agent's Transcriber by Measured Turn Latency > Three transcribers failed or lagged before landing on Soniox primary with Deepgram fallback. The 1.8 s average from one provider caused the agent to cut callers off nine times. - Author: Dil Ahdan (https://dil.dev/) - Published: 2026-10-16 - Updated: 2026-10-16 - Tags: voice-ai, transcription, latency, sip, multilingual - Canonical: https://dil.dev/notes/voice-agent-transcriber-latency/ ## Key takeaways - A missing API key returns silence, not an error, so you need a real call to confirm a transcriber is wired up. - 1.8 s average turn latency caused the agent to interrupt callers nine times and re-ask one confirmation three times. - SIP clips the first 1.7 s of agent audio; a sacrificial greeting absorbs the loss without cutting real content. - Deepgram nova-3 multi transcribes Urdu as Hindi, which matters on a line serving Punjabi, Hindi, Gujarati and Urdu speakers. - Soniox at 260 ms vendor median is the current primary, with Deepgram nova-3 multi as a live fallback. The short version: three transcribers failed or lagged before the line settled on Soniox as primary with Deepgram nova-3 multi as a fallback. The deciding metric was measured turn latency on real calls, not vendor marketing sheets. ## The Line and Why Transcriber Choice Is Hard Here The phone line serves callers in Punjabi, Hindi, Gujarati and Urdu. That combination rules out any transcriber that only handles English well or that collapses South Asian languages into one bucket. It also means accent variation is high and the cost of a mis-transcription is a confused caller who may not try again. The agent runs on VAPI. Transcriber config lives in the VAPI organisation settings, which means a missing key fails at the org level, not at the call level. ## Gladia: Silent Failure from a Missing Key The first transcriber tried was Gladia. Commit b9f6b08 shows the result: zero cost, zero tokens, no transcript. The call completed normally from the caller's perspective but the agent had nothing to work with. - **No error surfaced.** VAPI did not throw a visible exception. The call just produced an empty transcript. - **The diagnosis took a real call.** There is no dry-run that confirms a transcriber is actually wired up. You have to phone in. - **The fix was trivial once found.** No key existed in the VAPI org for Gladia. Adding one would have resolved it, but by that point the latency tests had already moved on. The lesson is not specific to Gladia. Any transcriber with a missing or malformed key will behave the same way. ## ElevenLabs scribe_v2_realtime: 1.8 s Average, Nine Interruptions Commits 522518c and handover b577e2a cover the session on 2026-10-06 with ElevenLabs scribe_v2_realtime. The numbers: - **Average turn latency: 1.8 s.** Max observed: 6.7 s. - **Agent cut callers off nine times** in the session because it assumed silence and started speaking. - **One confirmation was re-asked three times** because the transcriber had not finished returning the previous answer before the agent moved on. At 1.8 s average, the agent cannot tell the difference between a caller who has finished speaking and a caller who is mid-sentence but waiting for the transcriber to catch up. The agent's end-of-turn detection fires too early. This is not a tuning problem you can easily fix on the agent side without introducing a different problem: add too much silence padding and the agent feels sluggish on fast speakers. A parallel line running Deepgram on the same day averaged 0.46 s per turn with no interruptions logged. ## Deepgram nova-3 Multi: Fast but Urdu Lands as Hindi Deepgram nova-3 with language set to multi was the interim choice. The 0.46 s average is usable. The problem is language accuracy. - **Urdu is transcribed as Hindi.** The two languages share a large spoken vocabulary and the model collapses them. For a line where Urdu is a named supported language, that is a real accuracy gap. - **It is still the fallback.** The latency is measured and the failure mode is known, which makes it a reasonable safety net. - **nova-3 multi covers Punjabi and Gujarati** without obvious collapse, based on the calls reviewed. That part works. ## Soniox: Fast Median, but Balance Exhaustion Kills Calls Commit 950c5c5 records the first Soniox attempt. Every call failed at roughly 0.3 s with a `pipeline-error-soniox-transcriber-failed` event. The cause was a 402 `organisation_balance_exhausted` response. The account had no credit. - **The failure is fast and logged**, which is better than the silent Gladia key failure. You know immediately what happened. - **It still drops the call.** A caller hears nothing useful and hangs up. - **260 ms is Soniox's quoted vendor median**, not a figure from this line's calls. Once the account was funded, the line moved to Soniox primary in commit 3422569. The current config is Soniox primary, Deepgram nova-3 multi fallback. The fallback exists specifically because Soniox has a hard balance-exhaustion failure mode. If the account runs dry mid-day, Deepgram catches the calls. ## SIP Clipping and the Sacrificial Greeting On 2026-10-07, commits 2a73006 and fb594b2 address a separate problem that showed up on three calls: the opening agent message was clipped. - **SIP swallowed the first 1.7 s of agent audio** even after a deliberate 1 s pause before speaking. The caller heard the agent mid-sentence. - **The fix is a sacrificial phrase.** The first message now begins with something like "Hello, thanks for calling." That phrase carries no real information. If SIP clips it, the caller loses nothing meaningful. - **The real content starts after the sacrificial phrase.** The agent's actual greeting, the line name, the prompt for the caller's query, all of that comes after the throwaway opener. This is not a transcriber problem. It is a SIP path problem that affects what the caller hears, not what the transcriber receives. But it sits in the same set of commits because it was discovered during the same round of call testing. ## Picking a Transcriber Without a Benchmark Suite There is no formal benchmark suite here. The evaluation method was: - **Make real calls.** Log turn latency from the VAPI call records. - **Count interruptions.** If the agent cuts a caller off, that is a latency failure, not a personality quirk. - **Check language collapse.** Speak Urdu, read the transcript, see what language label comes back. - **Watch the account.** Soniox fails hard on zero balance. Set a balance alert before going live. The 260 ms Soniox vendor median is a useful prior but it is not a guarantee. Measure on your own line, with your own callers, in your own languages. ## What I'd do differently - **Test with a real key on the first call.** Do not assume a transcriber is connected until a transcript comes back. One test call before any other work would have caught the Gladia key issue immediately. - **Set a balance alert on any credit-based transcriber.** Soniox's failure mode is fast and logged, but it still drops calls. An alert at 20% balance remaining would give time to top up before callers are affected. - **Measure Urdu accuracy explicitly.** The Deepgram nova-3 multi Urdu-as-Hindi collapse was found by accident. A short scripted Urdu test phrase in the evaluation suite would have caught it on day one. - **Document the SIP clip length for the specific carrier.** 1.7 s is what this SIP path clips. A different carrier or SIP trunk may clip more or less. The sacrificial greeting length should be tuned to the actual clip, not a guess. ## Quick answers **Why did the first transcriber return nothing at all?** No API key existed in the VAPI organisation for that provider. The call completed, cost nothing and tokenised nothing, but produced no transcript. A missing key fails silently rather than throwing an obvious error, so you need at least one real call to confirm the integration is actually wired up. **What turn latency is acceptable for a phone voice agent?** On this line, 1.8 s average caused the agent to cut callers off nine times in a session. Deepgram on a parallel line averaged 0.46 s without that problem. Soniox quotes a 260 ms median. Anything above roughly 0.5 s starts to feel like the agent is ignoring the caller. **How does SIP audio clipping affect a voice agent's opening message?** SIP can swallow the first 1.7 s of agent audio even after a deliberate 1 s pause. The fix is a sacrificial phrase at the start of the first message, something like 'Hello, thanks for calling,' that carries no real information and can be lost without confusing the caller. **Does Deepgram nova-3 multi handle Urdu correctly?** Not reliably. On this line, Urdu speech was transcribed as Hindi. The two languages share a lot of spoken vocabulary, so the model collapses them. For a line where Urdu is a primary language, that is a meaningful accuracy gap to track. **What happens when a Soniox account runs out of balance?** Every call fails at around 0.3 s with a pipeline-error-soniox-transcriber-failed event and a 402 organisation_balance_exhausted response. The failure is fast and logged, which makes it easier to catch than a silent key error, but it still drops the call. **Why keep Deepgram as a fallback if it averaged 0.46 s and Soniox is faster?** Soniox has a balance-exhaustion failure mode that kills calls instantly. Deepgram nova-3 multi is a known quantity on this line with measured latency, so it acts as a safety net when the primary account runs dry or the Soniox service is unavailable.