Integrate Deepgram Flux with Twilio's Conversation Relay

Summary

Twilio's ConversationRelay connects phone calls to an AI stack while handling speech-to-text, text-to-speech, and the WebSocket session, leaving application code focused on the LLM and conversation logic. This tutorial updates a Node.js ConversationRelay application to use Deepgram Flux, which combines transcription and turn detection and is described as reducing response latency by up to 200–600 milliseconds and false interruptions by around 30 percent. It walks through cloning a Fastify demo, exposing it with ngrok, configuring environment variables, and adding Flux settings to the ConversationRelay TwiML, including eotThreshold, speechTimeout, and ignoreBackchannel. The supplied configuration uses an end-of-turn threshold of 0.8, a 3,000-millisecond speech timeout, and enabled backchannel ignoring, while the tutorial also explains partial prompts, smart formatting, and recognition hints.

Takeaways

1. 1

   Deepgram Flux combines transcription and turn detection in one model, with the tutorial describing up to 200–600 milliseconds lower response latency and around 30% fewer false interruptions.

2. 2

   Flux-only controls include eotThreshold, partialPrompts, and speechTimeout; eotThreshold ranges from 0.5 to 0.9, while speechTimeout accepts 600–5000 milliseconds or auto.

3. 3

   The tutorial leaves partialPrompts disabled because eager end-of-turn handling requires cancelling withdrawn LLM requests and duplicating responses when the final transcript matches.