Loading post…
TwilioPaul KampTutorial
Build an AI Voice Assistant with Twilio Voice and Media Streams, OpenAI's GPT-Live API, and Node.js
Summary
This tutorial shows how to build an AI voice assistant that answers phone calls using Twilio Programmable Voice, Twilio Media Streams, OpenAI's GPT-Live-1, and Node.js. GPT-Live-1 acts as the speech-to-speech model for listening, talking, and handling interruptions, while the delegated gpt-5.6-terra model handles deeper reasoning and tool use. The implementation uses a Fastify server with WebSocket support, dotenv configuration, a marin voice, and separate prompts for the voice and backend roles. Callers can request a city's top headline through the custom get_top_headline function or ask factual questions through OpenAI's hosted web_search tool. The guide exposes port 5050 with ngrok, points a Twilio number at an /incoming-call webhook, and concludes that the working example fits in under 200 lines.
Context
The tutorial is motivated by the launch of GPT-Live-1 and presents a way to make its speech-to-speech capabilities available through a phone call. It requires Twilio Voice access, GPT-Live-1 and gpt-5.6-terra API access, Node.js 22 or later, a public tunnel such as ngrok, and a phone for testing.
Approach / What changed
The implementation combines Twilio Programmable Voice and Media Streams with a Node.js Fastify server, WebSocket support, dotenv, and OpenAI's GPT-Live API. GPT-Live-1 handles the live voice interaction, while gpt-5.6-terra is delegated deeper reasoning and tool use. The assistant uses separate prompts, OpenAI's hosted web_search tool, and a custom get_top_headline function.
Takeaways
GPT-Live-1 handles audio interaction and interruptions, while gpt-5.6-terra is used for deeper reasoning and tool calls through delegation.
The example exposes a Node.js server on port 5050, uses ngrok to make it public, and configures a Twilio voice number to call the /incoming-call webhook.
The assistant supports both OpenAI's hosted web_search tool and a custom get_top_headline function that accepts a city parameter.
Source and freshness
- Source
- twilio.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.