Loading post…
TwilioXinghao HuangTutorial
How to Build a Voice AI Agent with OpenAI's GPT-Live, Twilio Agent Connect, and Python
Summary
This tutorial shows how to build a phone-based voice AI agent with OpenAI's GPT-Live, Twilio Agent Connect, and Python. It configures TAC's GPTLiveProvider to bridge Twilio Programmable Voice Media Streams to GPT-Live-1 in the OpenAI API, using 8kHz mu-law audio and avoiding custom WebSocket plumbing. The tutorial sets a proactive greeting, exposes a Python weather function through Responses delegation, and runs the function outside the real-time voice loop so a simulated three-second backend delay does not block the conversation. The resulting setup handles inbound calls through a Twilio webhook and outbound calls with an E.164 number, while the tutorial also notes per-call customization and alternative voice providers.
Context
The tutorial addresses the problem of voice AI demos that work in testing scripts but fall apart when connected to real phone calls and production traffic.
Approach / What changed
It uses Twilio Agent Connect's GPTLiveProvider with Twilio Programmable Voice Media Streams and GPT-Live-1, configures matching 8kHz mu-law audio, delegates tool decisions through the Responses API, and executes Python tools outside the real-time voice loop. A FastAPI server and Twilio webhook handle inbound calls, while the same VoiceChannel supports outbound calls.
Takeaways
GPT-Live is full-duplex and handles interruption logic; its audio format can be configured to match Twilio Media Streams' 8kHz mu-law wire format.
Responses delegation lets a model decide when to call a tool while Twilio Agent Connect executes the registered Python function outside the real-time voice loop.
The same GPTLiveProvider supports outbound calls through initiate_outbound_conversation, which requires the destination number in E.164 format.
Source and freshness
- Source
- twilio.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.