---
title: "How to Build a Voice AI Agent with OpenAI's GPT-Live, Twilio Agent Connect, and TypeScript"
description: "The post addresses the common problem of voice AI demos failing in production by introducing a solution using OpenAI’s GPT-Live and Twilio Agent Connect. GPT-Live is a full-duplex speech-to-speech model that separates conversation from background work, and Twilio Agent Connect provides a GPTLiveProvider to bridge Twilio Media Streams to GPT-Live without custom WebSocket plumbing. The tutorial walks through building a TypeScript application that connects inbound callers to a GPT-Live-powered voice agent, including setting up a Fastify server, configuring the session with a welcome instruction, and wiring tool calls via Responses delegation using a get_weather example. It also covers placing outbound calls using the same provider and demonstrates features like per-call customization and transcript logging. The result is a production-ready voice agent that can greet callers, answer questions, and execute backend functions, with the same pattern applicable to scenarios like appointment booking or order status lookups."
---

# How to Build a Voice AI Agent with OpenAI's GPT-Live, Twilio Agent Connect, and TypeScript

[Twilio](https://yomu.fyi/company/twilio) · Wen Zhu, Xinghao Huang · Oct 15, 2026

**Type:** Tutorial

## Summary

The post addresses the common problem of voice AI demos failing in production by introducing a solution using OpenAI’s GPT-Live and Twilio Agent Connect. GPT-Live is a full-duplex speech-to-speech model that separates conversation from background work, and Twilio Agent Connect provides a GPTLiveProvider to bridge Twilio Media Streams to GPT-Live without custom WebSocket plumbing. The tutorial walks through building a TypeScript application that connects inbound callers to a GPT-Live-powered voice agent, including setting up a Fastify server, configuring the session with a welcome instruction, and wiring tool calls via Responses delegation using a get\_weather example. It also covers placing outbound calls using the same provider and demonstrates features like per-call customization and transcript logging. The result is a production-ready voice agent that can greet callers, answer questions, and execute backend functions, with the same pattern applicable to scenarios like appointment booking or order status lookups.

## Context

The post provides a tutorial for building a production-ready voice AI agent using OpenAI’s GPT-Live and Twilio Agent Connect. It addresses the common failure of voice AI demos in production by offering a declarative config interface that bridges Twilio Programmable Voice Media Streams to GPT-Live without custom WebSocket code. The tutorial covers inbound and outbound calling, tool calls via Responses delegation, and features like welcome instructions and per-call customization.

## Approach / What changed

Use Twilio Agent Connect’s GPTLiveProvider to bridge Twilio Media Streams to GPT-Live. Set up a VoiceChannel with a GPTLiveProviderConfig containing defaultSessionConfig (model, instructions, audio format) and welcomeInstruction. Define tools using defineTool() and register them in the provider config and delegation.responses.tools. Use TACServer to expose a TwiML endpoint. For outbound calls, call voiceChannel.initiateOutboundConversation().

## Takeaways

- GPT-Live is a full-duplex speech-to-speech model that separates conversation from background work, allowing tool execution outside the real-time voice loop.
- Twilio Agent Connect’s GPTLiveProvider bridges Twilio Media Streams to GPT-Live using a declarative config interface, eliminating the need for custom WebSocket plumbing.
- Tool calls are delegated to a Responses API model, and the actual TypeScript function is executed by Twilio Agent Connect, which then speaks back the result.

## FAQ

### How do I set up a welcome message that speaks first?

Set the welcomeInstruction property in GPTLiveProviderConfig. The agent will speak the provided text immediately upon connection without waiting for the caller.

### How do I handle tool calls with GPT-Live?

Define tools using defineTool() and register them in the provider config's tools array and in delegation.responses.tools. The model delegates tool calls to a Responses API model, and Twilio Agent Connect executes the TypeScript function and speaks the result.

### How do I place an outbound call?

Use voiceChannel.initiateOutboundConversation({ to: '+15551234567' }) with the number in E.164 format. TAC handles placing the call, connecting the Media Stream, and bridging to GPT-Live.

**Tags:** [OpenAI](https://yomu.fyi/topic/openai), [Tool Calling](https://yomu.fyi/topic/tool-calling), [Twilio](https://yomu.fyi/topic/twilio), [TypeScript](https://yomu.fyi/topic/typescript), [Voice AI](https://yomu.fyi/topic/voice-ai)

- Source: [Twilio](https://www.twilio.com/en-us/blog/developers/tutorials/integrations/tac-gpt-live-voice-ai-agent-typescript)
- Source URL: https://www.twilio.com/en-us/blog/developers/tutorials/integrations/tac-gpt-live-voice-ai-agent-typescript
- Ingested by Yomu: 2026-10-16T00:00:24.447Z

[Read original post](https://www.twilio.com/en-us/blog/developers/tutorials/integrations/tac-gpt-live-voice-ai-agent-typescript)