The short version: three transcribers failed or lagged before the line settled on Soniox as primary with Deepgram nova-3 multi as a fallback. The deciding metric was measured turn latency on real calls, not vendor marketing sheets.
The Line and Why Transcriber Choice Is Hard Here
The phone line serves callers in Punjabi, Hindi, Gujarati and Urdu. That combination rules out any transcriber that only handles English well or that collapses South Asian languages into one bucket. It also means accent variation is high and the cost of a mis-transcription is a confused caller who may not try again.
The agent runs on VAPI. Transcriber config lives in the VAPI organisation settings, which means a missing key fails at the org level, not at the call level.
Gladia: Silent Failure from a Missing Key
The first transcriber tried was Gladia. Commit b9f6b08 shows the result: zero cost, zero tokens, no transcript. The call completed normally from the caller's perspective but the agent had nothing to work with.
- No error surfaced. VAPI did not throw a visible exception. The call just produced an empty transcript.
- The diagnosis took a real call. There is no dry-run that confirms a transcriber is actually wired up. You have to phone in.
- The fix was trivial once found. No key existed in the VAPI org for Gladia. Adding one would have resolved it, but by that point the latency tests had already moved on.
The lesson is not specific to Gladia. Any transcriber with a missing or malformed key will behave the same way.
ElevenLabs scribe_v2_realtime: 1.8 s Average, Nine Interruptions
Commits 522518c and handover b577e2a cover the session on 2026-10-06 with ElevenLabs scribe_v2_realtime. The numbers:
- Average turn latency: 1.8 s. Max observed: 6.7 s.
- Agent cut callers off nine times in the session because it assumed silence and started speaking.
- One confirmation was re-asked three times because the transcriber had not finished returning the previous answer before the agent moved on.
At 1.8 s average, the agent cannot tell the difference between a caller who has finished speaking and a caller who is mid-sentence but waiting for the transcriber to catch up. The agent's end-of-turn detection fires too early. This is not a tuning problem you can easily fix on the agent side without introducing a different problem: add too much silence padding and the agent feels sluggish on fast speakers.
A parallel line running Deepgram on the same day averaged 0.46 s per turn with no interruptions logged.
Deepgram nova-3 Multi: Fast but Urdu Lands as Hindi
Deepgram nova-3 with language set to multi was the interim choice. The 0.46 s average is usable. The problem is language accuracy.
- Urdu is transcribed as Hindi. The two languages share a large spoken vocabulary and the model collapses them. For a line where Urdu is a named supported language, that is a real accuracy gap.
- It is still the fallback. The latency is measured and the failure mode is known, which makes it a reasonable safety net.
- nova-3 multi covers Punjabi and Gujarati without obvious collapse, based on the calls reviewed. That part works.
Soniox: Fast Median, but Balance Exhaustion Kills Calls
Commit 950c5c5 records the first Soniox attempt. Every call failed at roughly 0.3 s with a pipeline-error-soniox-transcriber-failed event. The cause was a 402 organisation_balance_exhausted response. The account had no credit.
- The failure is fast and logged, which is better than the silent Gladia key failure. You know immediately what happened.
- It still drops the call. A caller hears nothing useful and hangs up.
- 260 ms is Soniox's quoted vendor median, not a figure from this line's calls. Once the account was funded, the line moved to Soniox primary in commit 3422569.
The current config is Soniox primary, Deepgram nova-3 multi fallback. The fallback exists specifically because Soniox has a hard balance-exhaustion failure mode. If the account runs dry mid-day, Deepgram catches the calls.
SIP Clipping and the Sacrificial Greeting
On 2026-10-07, commits 2a73006 and fb594b2 address a separate problem that showed up on three calls: the opening agent message was clipped.
- SIP swallowed the first 1.7 s of agent audio even after a deliberate 1 s pause before speaking. The caller heard the agent mid-sentence.
- The fix is a sacrificial phrase. The first message now begins with something like "Hello, thanks for calling." That phrase carries no real information. If SIP clips it, the caller loses nothing meaningful.
- The real content starts after the sacrificial phrase. The agent's actual greeting, the line name, the prompt for the caller's query, all of that comes after the throwaway opener.
This is not a transcriber problem. It is a SIP path problem that affects what the caller hears, not what the transcriber receives. But it sits in the same set of commits because it was discovered during the same round of call testing.
Picking a Transcriber Without a Benchmark Suite
There is no formal benchmark suite here. The evaluation method was:
- Make real calls. Log turn latency from the VAPI call records.
- Count interruptions. If the agent cuts a caller off, that is a latency failure, not a personality quirk.
- Check language collapse. Speak Urdu, read the transcript, see what language label comes back.
- Watch the account. Soniox fails hard on zero balance. Set a balance alert before going live.
The 260 ms Soniox vendor median is a useful prior but it is not a guarantee. Measure on your own line, with your own callers, in your own languages.
What I'd do differently
- Test with a real key on the first call. Do not assume a transcriber is connected until a transcript comes back. One test call before any other work would have caught the Gladia key issue immediately.
- Set a balance alert on any credit-based transcriber. Soniox's failure mode is fast and logged, but it still drops calls. An alert at 20% balance remaining would give time to top up before callers are affected.
- Measure Urdu accuracy explicitly. The Deepgram nova-3 multi Urdu-as-Hindi collapse was found by accident. A short scripted Urdu test phrase in the evaluation suite would have caught it on day one.
- Document the SIP clip length for the specific carrier. 1.7 s is what this SIP path clips. A different carrier or SIP trunk may clip more or less. The sacrificial greeting length should be tuned to the actual clip, not a guess.