Loading…
Topic
WebSockets
25 posts about WebSockets. Every summary links to the original.
- 25 posts
- 5 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
How to Orchestrate Multi-Call Conversations with an LLM and Twilio Conversation Memory
The tutorial builds a C# .NET 9 ASP.NET Core voice service that preserves caller context, preferences, and action history across separate inbound calls. It combines Twilio Conversation Relay and a Twilio Conversation Memory Store with an OpenAI gpt-4o-mini service, using WebSockets to connect the call flow and stream generated text. The setup covers creating the project, installing Twilio, DotNetEnv, and OpenAI packages, configuring credentials and IDs in .env, exposing local webhooks with ngrok, and routing /voice to /ws. The sample assistant is framed as Owlbert's Auto Repair and is instructed to speak naturally, avoid lists and emojis, refer to caller details, and acknowledge uncertainty rather than guess. Testing uses two calls from the same number; the expected result is that the second call recalls first-call details and that Twilio shows a stored conversation summary.
How to Connect Your Twilio Agent to External APIs
This tutorial shows how to connect a Twilio voice agent to an external REST API so it can retrieve live information during a conversation. It uses C# .NET 9, Twilio Conversation Relay, WebSockets, OpenAI tool calling, and the authentication-free Cat Facts API as a demonstration. An ASP.NET Core application exposes a /voice webhook, connects calls to a /ws endpoint, maintains conversation messages, streams model responses, and executes get_cat_fact by requesting and parsing JSON from the Cat Facts service. The guide also covers environment variables, NuGet dependencies, ngrok tunneling, Twilio number configuration, and troubleshooting for webhook URLs and API keys. After setup, callers can ask for a cat fact and receive one from the external service, illustrating how function tools can give voice agents real-time data and perform API-backed work.
How to Connect Your Twilio Agent to External APIs in Node.js
Voice agents can converse, but they need external APIs to retrieve live information or take actions such as checking inventory or booking appointments. This tutorial builds a Node.js voice agent with Twilio Conversation Relay, Express, WebSockets, and the OpenAI Node.js SDK. A Conversation Relay WebSocket handler collects voice prompts, maintains conversation messages, streams model output back to the call, and defines a get_cat_fact tool that fetches JSON from the unauthenticated Cat Facts REST API. The service returns the fetched fact or an error message, while TwiML connects Twilio's voice-capable number to the WebSocket endpoint. Testing uses Node.js 18+, an OpenAI API key, ngrok, and a POST /voice webhook; the stated result is that callers can ask for a cat fact and hear the API response.
How to Orchestrate Multi-Call Conversations with an LLM and Twilio Conversation in Node.js Memory
Disconnected voice calls can force callers to repeat information, creating friction for AI phone agents. It builds a Node.js Express service that preserves caller context, preferences, and action history across separate inbound calls using Twilio Conversation Memory. The implementation uses Express and express-ws for HTTP and WebSocket handling, the OpenAI SDK with gpt-4o-mini for streamed responses and summaries, and axios for Twilio’s Conversation Memory REST API; a memory store ID and other credentials are configured through environment variables. The sample application connects Twilio’s Conversation Relay to an auto-repair assistant, retrieves prior context for recognized callers, and logs a summary when a call ends. Testing involves two calls from the same phone number to verify that the second call recalls details from the first.
How to Connect Your Twilio Agent to External APIs with PHP
The tutorial shows how to connect a Twilio voice agent to external REST APIs so an LLM-driven conversation can retrieve live information or support actions such as checking inventory or booking appointments. It builds the example in PHP 8.5 or later with OpenSwoole, Twilio Conversation Relay, OpenAI, and a WebSocket server, using Cat Facts as an unauthenticated API. The application exposes a POST /voice webhook that returns TwiML and connects Conversation Relay to a wss /ws endpoint, while session handling passes caller prompts into the OpenAI service. Tool calling routes the request to the Cat Facts endpoint, parses the JSON fact field, and returns either the fact, “No fact returned.”, or an error when the API response is unsuccessful or unavailable. Testing requires running php server.php, exposing port 8080 with ngrok, and configuring the Twilio number’s incoming webhook to /voice; the conclusion says external HTTP endpoints give agents real-time data.
How to Orchestrate Multi-Call Conversations with an LLM and Twilio Conversation Memory with PHP
This tutorial shows how to build a PHP voice agent that retains caller context, preferences, and action history across separate inbound calls. It uses Twilio Conversation Memory with a Memory Store, an Open Swoole service, Twilio ConversationRelay, and an OpenAI integration using gpt-4o-mini; the example models an auto-repair shop assistant. The OpenAI service appends supplied prior customer context to its system prompt, while streamed responses are sent through a WebSocket and environment variables provide API credentials and the Memory Store ID. After running the service on port 9501, exposing it with ngrok, and configuring the phone-number webhook, the tutorial tests two calls from the same number and says the second call should remember details from the first.
Building resilient real-time streaming workers with Amazon DynamoDB leases
Managing hundreds of persistent outbound WebSocket connections requires explicit ownership because a failed worker can drop connections and stop data flow until they are reassigned. The described fleet-management pattern uses Amazon DynamoDB leases, conditional writes, and a global secondary index to ensure one worker owns each connection, renew ownership with heartbeats, and find expired leases for reconciliation. Amazon ECS on AWS Fargate workers poll Amazon SQS for notifications, maintain the WebSockets, and release or transfer ownership during graceful shutdown, failures, and scaling. The design also records desired state, the upstream URL, and the last processed sequence number in the lease item, while clock synchronization supports its time-based expiry. The post identifies heartbeat writes as the dominant DynamoDB cost and recommends provisioned capacity with Auto Scaling for sustained high connection counts, while noting that longer intervals reduce cost but extend failover.
Build an AI Voice Assistant with Twilio Voice and Media Streams, OpenAI's GPT-Live API, and Node.js
This tutorial shows how to build an AI voice assistant that answers phone calls using Twilio Programmable Voice, Twilio Media Streams, OpenAI's GPT-Live-1, and Node.js. GPT-Live-1 acts as the speech-to-speech model for listening, talking, and handling interruptions, while the delegated gpt-5.6-terra model handles deeper reasoning and tool use. The implementation uses a Fastify server with WebSocket support, dotenv configuration, a marin voice, and separate prompts for the voice and backend roles. Callers can request a city's top headline through the custom get_top_headline function or ask factual questions through OpenAI's hosted web_search tool. The guide exposes port 5050 with ngrok, points a Twilio number at an /incoming-call webhook, and concludes that the working example fits in under 200 lines.
Make Outbound Calls with Twilio Voice and Media Streams, GPT-Live-1 in the OpenAI API, and Node.js
This tutorial shows how to build an outbound AI voice agent that uses Twilio Programmable Voice and Media Streams to call a permitted phone number, rather than waiting for inbound calls. It connects the call to GPT-Live-1 in the OpenAI API through a Node.js server, with Fastify, ws, dotenv, and the Twilio helper library handling the application and relay. The implementation reads the destination from a --call flag, validates E.164 formatting, exposes the server through a public tunnel, and uses environment variables for credentials and the calling number. Owlie identifies itself as an AI assistant, retrieves a callback note with get_callback_reason, delegates factual questions to hosted web_search, and can be interrupted during speech. The completed demo rings the chosen number, explains why it is calling, and can return random callback notes or web-searched answers, while the tutorial stresses verified numbers, model access, and outbound-calling compliance.
Choosing a low-latency infrastructure layer for conversational AI
Conversational AI call-center latency can feel like dead air, and the delay may come less from LLM inference than from the infrastructure moving audio among telephony, media servers, ASR, the agent, and TTS. The described approach uses a unified voice path: ConversationRelay provides a WebSocket API to stream live call audio, handle speech-to-text and text-to-speech in one connected pipeline, connect with a backend agent, and support barge-in. It is model-agnostic, allowing teams to retain their LLM, knowledge bases, AI tools, and third-party voice providers, while Conversation Orchestrator, Conversation Memory, and Agent Connect address context and integration. Twilio’s internal benchmarks report median latency below 0.5 second and below 0.725 second at the 95th percentile, though relevance and context also matter alongside speed.
Build a real-time market data app with ClickHouse and Massive
The post presents a real-time market-data application that ingests trade and quote ticks from Massive and stores them in ClickHouse for low-latency querying and visualization. It distinguishes quotes, which contain bid and ask prices and sizes, from trades, which record executed price, size, timestamps, and related fields. The implementation uses an authenticated Massive WebSocket subscription, Node.js ingestion, separate ClickHouse MergeTree tables ordered by symbol and one-minute timestamp buckets, and React charts. It also describes materialized-view aggregation for visualization and supplies a runnable repository with setup, build, start, admin, and synthetic-seed paths; the conclusion recommends durable delivery, replay, authentication, and workload-specific tuning before production use.
Add an AI Voice Assistant to a Twilio Video Room with Conversation Relay
The tutorial addresses wasted patient time in telehealth waiting rooms by adding an AI voice assistant to a Twilio Video room for intake collection. It combines Twilio Programmable Video, Conversation Relay, and an LLM, with the assistant joining as an audio-only participant and handling questions about visit reasons, symptoms, and medications. A two-leg Voice call bridges the room to Conversation Relay: one leg participates in the room, while the other sends transcribed audio to a WebSocket server and returns spoken LLM responses. The tutorial uses Node.js, Express, the Twilio helper library, OpenAI, ws, dotenv, and ngrok, and explains setup, browser interaction, and troubleshooting. Its stated result is a working application in which a patient converses naturally with the AI before a provider joins, with GPT-4o-mini processing speech.
SignalR Scalability: Scaling Real-Time SignalR Applications on Heroku
Scaling a SignalR application beyond one server requires coordination because each server maintains its own connected-client list. The guide builds a .NET 9.0 chat application, adds the Microsoft.AspNetCore.SignalR.StackExchangeRedis package, and configures Redis as a backplane with a SignalRChat channel prefix and environment-specific connection settings. It deploys the application to Heroku, changes the dyno type to standard-1x, and runs three web dynos; without session affinity, WebSocket startup fails because the connection may be routed to a different dyno. Enabling Heroku’s http-session-affinity feature keeps requests on one dyno, after which both browsers establish stable WebSocket connections and exchange chat messages successfully across the scaled deployment.
VSCode’s SSH Agent Is Bananas
VSCode’s remote-editing flow over SSH is examined as a concern for running LLM-driven agent workflows on remote machines. The motivation is to keep iterative code generation and execution away from a development laptop, ideally on a clean-slate Linux instance that starts instantly and cannot damage the host. Unlike Emacs Tramp, which operates through commands on the remote connection, VSCode uses a Bash snippet stager to download an agent, including a binary Node installation. That agent communicates over port-forwarded SSH through a WebSockets connection to the VSCode front end and can browse the filesystem, edit arbitrary files, launch shell PTYs, and persist itself. The author says this raises concerns for development servers and especially production incident work, while noting a custom Fly Machine connection avoids needing this mechanism.
Vanilla with Candy Sprinkles
Vanilla with Candy Sprinkles presents a baseline full-stack Node.js application: a stateful visitor counter that increments across refreshes, tabs, browsers, and machines. The post starts with a dependency-free package.json and a 72-line server.js, then uses node-demo to generate variants with ECMAScript modules, EJS or Mustache templates, and Express. It adds SQLite3 or PostgreSQL for persistence, Tailwind CSS and TypeScript with a build step, and Drizzle, Knex, or Prisma as ORM options, noting differing language and build requirements. WebSockets provide real-time updates, htmx can handle client-side connection and DOM work, and Redis coordinates notifications across multiple servers; Fly.io deployment can scale the setup across eight instances in four regions. The concluding section points to further alternatives, including React, Angular, Lit, SolidJS, Svelte, Vue, esbuild, rollup, and other deployment targets, while inviting alternate implementations.
Logbook: November 14 to December 5, 2022
An edition of Fly’s Logbook covering November 14 to December 5, 2022 collects updates and tutorials for building and operating applications on Fly. It features a real-time collaboration example that connects Replicache with WebSockets and Fly-Replay, plus a LiveWire walkthrough for streaming server content to browsers and another for unloading data during large-dataset pagination. A separate guide explains how to run Supercronic from Fly Dockerfiles when precise cron timing is needed beyond Fly Machines’ monthly, weekly, daily, and hourly scheduling options. The edition also recounts how Fly Postgres was built, despite Fly’s stated preference for off-platform database services such as RDS, CrunchyData, and PlanetScale. The post notes that the update followed a longer break while many Fly staff spent time with family over Thanksgiving.
Real-Time Collaboration with Replicache and Fly-Replay
Distributed real-time collaboration becomes difficult when a single server coordinating document changes is overloaded and documents must be split across machines without exposing routing complexity to clients. This demo combines Replicache, a custom WebSocket endpoint, Fly Machines, and the Fly-Replay header to route each document’s connection through a shared router to the machine responsible for that document. The server keeps connections by spaceID, sends initial and incremental state, and tracks messages already delivered; the client overrides Replicache’s push and pull handlers to exchange data through the persistent socket. The resulting application has React frontend, Express router, and Replicache/WebSocket backend layers, with each todo list assigned to one backend machine. The author cautions that the approach is demonstrated only in a basic demo and should receive serious production testing and tuning.
How LiveView Took on a Life of Its Own
Phoenix LiveView began as an attempt to build dynamic server-rendered applications without writing JavaScript, using the server for interactions such as realtime reservation updates. Its initial design used stateful Phoenix Channels, a WebSocket connection, client event bindings, and full template re-renders sent back as HTML, a model that worked but produced redundant server work and large payloads. Adding Phoenix PubSub let LiveViews subscribe to reservation events and update all connected browsers through ordinary Elixir callbacks, without client changes. The implementation then evolved around diffing dynamic values, shared static template references, and cached live component templates, reducing transferred data and avoiding growing client bundles. The post concludes that this programming model sidesteps HTTP for dynamic features, with compact payloads and less client-side code.
LiveBeats: Building a social music app with Phoenix LiveView
LiveBeats is an open-source social music application built as a full-stack Phoenix reference app and learning test bed for LiveView features. It demonstrates a shared real-time UI for playlist playback, presence, GitHub OAuth, and concurrent MP3 uploads, using Phoenix PubSub and the existing LiveView WebSocket connection rather than separate REST, JSON, or polling layers. Uploads are processed concurrently in temporary files to verify MP3 validity and enforce a duration below 20 minutes, while HEEx function components and slots provide reusable UI and JS commands keep immediate interactions such as modal hiding on the client. The application runs across five continents with automatic clustering, file proxying, and regional Postgres read replicas; the post presents this as a way to deliver responsive global behavior with relatively minor changes.
Launching Livebook using LiveView
Fly.io’s Livebook launcher uses a single LiveView process to deploy a Livebook instance, but temporarily held deployment secrets could be lost when a user refreshed the browser or the server restarted. The secrets, including liveview_password, were sent to deployment and stored in Vault, so the values could not be retrieved; keeping them in a database or Redis also conflicted with avoiding persistent server-side storage. The implementation encrypts the private state with Phoenix.Token.encrypt/4, stores the resulting blob in browser SessionStorage, and uses LiveView hooks and websocket events to restore it through Phoenix.Token.decrypt/4. After recovery, the UI resumes tracking deployment progress, then clears the cached data when deployment succeeds or fails, while closing the tab also clears SessionStorage.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.