AG-UI as the Agent–UI Contract Layer
AG-UI is a typed event contract between an agent backend and a frontend, turning streamed agent state into UI. This diagram maps the layers and shows where human review plugs in.
Loading…
Topic
Streaming
123 posts about Streaming. Every summary links to the original.
- 123 posts
- 18 companies
- 7 explainers
See the idea
Streaming explainers
Streaming ASR vs. Batch ASR: The Buffer-Latency Trade-off
Streaming and batch ASR pipelines compared: how input buffer length, speaker diarization, and where audio is processed trade latency against transcript quality and privacy.
When the Pipeline Pushes Back
Backpressure is a feedback loop that slows work when downstream capacity or health deteriorates. Compare its gains, costs, and signals for choosing a control point.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Automatic rule backtesting with large quantities of data
Evaluating new or modified risk rules previously required Grab analysts to run slow offline Presto queries, manually construct payloads, or run rules in shadow mode for days. To standardize and accelerate this workflow, Grab developed an automated backtesting system powered by an AWS EMR Spark pipeline. Historical events are continuously ingested via Kafka and a Kubernetes stream pipeline into S3 using Snappy-compressed Parquet. Users configure replay intervals and rule definitions directly in the rule engine UI, which triggers asynchronous Spark jobs through Amazon SQS and Lambda. This automated simulation replaces multi-week shadow mode runs and generates downloadable aggregation metrics on transactions, user counts, and treatment outcomes.
How Kafka Connect helps move data seamlessly
Grab's real-time data platform team, Coban, implemented a managed Kafka Connect ecosystem on Kubernetes to streamline moving data in and out of Apache Kafka. To resolve dual-write consistency issues and capture pre- and post-change data, the team integrated Debezium connectors to capture MySQL binlog events and accommodate database DDL migrations. For disaster recovery and stream migrations, Coban deployed MirrorMaker2 connectors managed via Terraform to handle message mirroring and consumer offset translation across AWS regions. Additionally, they developed a custom converter utilizing Confluent Schema Registry to transform Protobuf-serialized Kafka records into JSON for ingestion into Azure Event Hubs. This architecture enabled zero-downtime cluster migrations and robust cross-region disaster recovery.
Supabase Realtime, with Multiplayer Features
Supabase announced the third major release of Supabase Realtime, evolving the system from a single-tenant PostgreSQL change listener into a multi-tenant, distributed real-time platform. Built using Elixir and the Phoenix framework across a distributed node cluster, the new architecture decouples the database listener into an extension model and opens generic WebSocket functionality. New multiplayer capabilities include Presence, which uses Conflict-free Replicated Data Types (CRDTs) to track user connection states, and Broadcast, which distributes ephemeral messages across topics without writing to disk. The modular extension design allows listening to multiple PostgreSQL instances, external systems like MySQL, and financial or blockchain feeds, as demonstrated on a globally distributed 20-node cluster.
Real-time data ingestion in Grab
Service teams at Grab historically had to dual-write transactional data into databases and Kafka, creating data integrity issues during transaction failures alongside substantial schema maintenance overhead. To overcome these limitations and eliminate burst reads from SQL-based queries, the Caspian team built a real-time ingestion platform synchronising MySQL, Aurora, and DynamoDB directly to Kafka. For MySQL and Aurora, the platform uses Debezium with Kafka Connect on ROW-format binlogs, while DynamoDB changes are captured via DynamoDB streams with auto-scaling AWS Lambda functions. Messages encoded in Protobuf are transported via Kafka and ingested into Amazon S3 using a Golang stream processor. This architecture supports search indexing in Elasticsearch, automated data lake pipelines, cross-region disaster recovery replication, and audit trails.
Exposing a Kafka Cluster via a VPC Endpoint Service
To replace VPC peering and reduce attack surfaces, Grab exposed a multi-Availability Zone Apache Kafka cluster in its main AWS VPC to clients in a separate GrabKios VPC using AWS VPC Endpoint Service. Because Kafka requires clients to establish deterministic connections to individual brokers, the team configured a Network Load Balancer with unique TCP ports and dedicated target groups for each broker alongside a shared bootstrap port. They added custom listeners on the Kafka brokers to advertise endpoints using private Route 53 CNAMEs rather than raw interface hostnames. To eliminate unnecessary cross-AZ network latency and data transfer costs, the architecture was refined to advertise AZ-specific private CNAMEs mapped directly to zonal endpoint interfaces.
Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers
Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(n²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming.
Supabase Beta November 2021: Launch Week Recap
Supabase celebrated its third Launch Week and one year in beta by rolling out several major infrastructure and tooling updates. The company open-sourced its Dashboard for self-hosting and integrated Logflare to provide searchable database and API logs directly in the interface. For database querying and APIs, Supabase introduced a Postgres extension that resolves each GraphQL request with a single SQL statement to minimize network IO overhead, while upgrading its default hosted database to PostgreSQL 14 and deploying PostgREST 9.0. Security capabilities were expanded by enabling Row Level Security policies across the Realtime API to restrict streams and subscriptions on a per-user basis. Additional releases during the week included faster media file delivery, an open-sourced launch methodology, and a free course combining Next.js, Stripe, and Supabase.
Scaling Shopify's BFCM Live Map: An Apache Flink Redesign
Shopify’s 2021 BFCM live map had to scale from more than 1.7 million merchants while adding product trends and unique shoppers, after the prior Cricket-and-Redis backend showed volume and delivery weaknesses. The redesign placed Apache Flink on the critical path to filter irrelevant checkout events, reducing Cricket’s workload to about one percent of event volume, while cross-region sharding and active-active deployment supported availability. Flink computed new metrics, including 500 product categories ranked by sales-volume change every five minutes, while Cricket retained existing metric computation and deduplication. In load tests, the pipeline handled 1 million checkout events per second, and its jobs ran at 100 percent uptime during BFCM without backpressure or manual intervention. Redis still caused high CPU and memory pressure, producing brief arc-visual errors for a small sample of users, so Shopify plans a simpler Flink-centered design.
Realtime Postgres RLS now available on Supabase
Supabase updated its Realtime server to enforce PostgreSQL Row Level Security (RLS) policies when broadcasting database changes over websockets. Previously, Realtime operated as an opt-in beta feature that sent all replication changes to every client regardless of user authorization. To enforce RLS per subscriber without heavy performance overhead, Supabase introduced WALRUS, a security engine colocated inside PostgreSQL. For each replication change, WALRUS looks up active subscribers, assumes their identities, and evaluates row visibility using prepared statements queried by primary key. This in-database evaluation avoids external network round trips and single-query planning overhead while returning an authorized subscriber list to Realtime.
Hosting your Models and Datasets on Hugging Face Spaces using Streamlit
Streamlit enables developers to create interactive web demonstrations and visualizations for machine learning models and datasets. In an illustrative text completion interface recreating Write with Transformer, UI components such as text areas, sidebars, number inputs, and continuous sliders collect user text and inference hyperparameters. For data inspection, the streaming feature in Hugging Face Datasets loads large corpora directly into memory without requiring complete local downloads, integrating with pandas, matplotlib, and seaborn for tabular and chart rendering. Deploying these interactive applications to Hugging Face Spaces involves uploading the code files while defining dependencies inside a requirements.txt file. Matching local Streamlit versions ensures consistent runtime execution when hosting models and datasets online.
Automating Multi-Armed Bandit testing during feature rollout
Traditional feature rollouts and Multi-Armed Bandit testing operate as separate workflows that often depend on delayed offline analysis. To eliminate manual intervention, the Multi-Armed Bandit Optimiser automates testing concurrently during feature rollouts by responding to minute-level feedback metrics. The architecture connects Kafka Streams data processing, a metrics server with Spark jobs, and an adaptive rollout module updating online experimentation configurations. Candidate models are evaluated via Thompson Sampling on Beta distributions, with Monte Carlo simulations determining traffic allocation across user entities. In production for the GrabFood recommendation widget, the system optimizes the Effective Conversion Rate over a 30-minute window and includes fallback distribution logic.
Supabase Beta July 2021
Supabase concluded July 2021 with its second Launch Week event, delivering daily product updates and infrastructure upgrades across its platform. Authentication features expanded to support SMS logins through Twilio, Twitch authentication, and API endpoints for generating custom email authentication links. Storage capabilities were upgraded to include media streaming, public buckets, directory uploads, and speed improvements, alongside dashboard enhancements for usage statistics and database insights. Furthermore, all newly created Supabase projects now run PostgreSQL 13.3 by default, and instances incorporate the updated PostgREST version 8.0 server to generate CRUD APIs. The release also introduced preliminary components for Supabase functions as development continues toward general availability.
Open Source Hackathon Winners
Supabase announced the winners and runners-up for its inaugural open-source hackathon after concluding deliberations across a larger-than-expected pool of entries. Top entries included Work From, an employee schedule tracking application, and Fireplace, a video streaming platform that leverages Supabase Storage to synchronize playback events across multiple viewers. Additional recognized projects encompassed Feature.so for modular application component management, Flutter Add-ons for telemetry and crash reporting, and Peckish Peach, which uses machine learning to generate recipes from input ingredients. All submitted projects remain fully open source for community inspection, contribution, and forking. Gold and silver prize winners receive custom limited-edition Supabase apparel, while every hackathon participant receives a participation prize.
Supabase Storage now in Beta
Supabase released Storage Beta with several functional and architectural improvements three months after its alpha launch. The storage service now supports the Content-Range HTTP header for streaming audio and video, alongside entirely public buckets configured with dashboard safeguards and standard authorization policies. Directory uploads are enabled via the File and Directory entries API, while self-hosted instances can use a new local filesystem backend rather than requiring AWS S3 configuration. To support Node.js without adding polyfills or bundling complexity, the storage server was updated to accept binary data and streams directly. Additionally, the dashboard storage explorer simulates filesystem directories using hidden placeholder files and implements list virtualization.
Capturing Every Change From Shopify’s Sharded Monolith
Shopify’s separate internal warehouse and Merchant Analytics Platform created duplicated work, differing extraction results, and incompatible tooling while demand grew for fresher, low-latency merchant data. Longboat, its query-based CDC service, polls sharded-monolith replicas in batch jobs using updated_at, but queries can run only hourly and cannot detect hard deletes or updates that bypass that field. Shopify therefore adopted Kafka and Debezium CDC to capture database changes, standardize extraction, and make the event stream usable by both streaming and batch consumers. CDC-backed Beam modeling reduced average freshness for one migration from one day to one hour, while Kafka became a shared source for operational and analytical data. Shopify still identifies incremental snapshotting and streaming-state problems, including reconciling MySQL table chunks with ongoing binlog events without expensive locks, as ongoing work.
Apache Beam for Search: Getting Started by Hacking Time
Search relevance can use clickstream signals such as clicks and purchases to promote popular results and demote results users do not favor, but batch processing can delay those adjustments by days. The Discovery team at Shopify uses Apache Beam, a unified batch and stream processing system, to support real-time and historical search-behavior workflows, including boosting, popularity aggregates, offline evaluation sets, and reinforcement learning tasks. The article’s central lesson is that Beam requires careful handling of event time, processing time, delayed events, out-of-order data, and watermarks, especially when combining streams. A Kafka source can use Kafka processing time, client create time with a five-minute out-of-order allowance, or a custom timestamp policy that reads the SearchQueryEvent.searchTimestamp field; Beam windows add further buffering and lateness controls. Accurate timestamping is presented as the first milestone for replaying clickstream data reliably before tackling more complex operations such as streaming joins.
Optimally Scaling Kafka Consumer Applications
Grab's Coban platform runs Golang-based stream processing pipelines on Kubernetes, servicing roughly 400 billion events weekly from Kafka. The initial Horizontal Pod Autoscaler setup caused resource waste and uneven load distribution across Kafka partitions during scale-in and scale-out events. To resolve this, Grab moved to a fixed pod count matching the topic's partition count and adopted Vertical Pod Autoscaling, reducing resource usage versus requests by approximately 45%. The team also introduced Kubernetes priority classes to segment latency-sensitive workloads onto On-Demand nodes and non-critical jobs onto Spot instances. Additionally, overprovisioning via low-priority placeholder pods managed by Cluster Proportional Autoscaler enabled rapid pod rescheduling and reduced deployment delays.
Plumbing At Scale
Grab's backend services process terabytes of data ingress per hour, generating recurring needs for stream transformations, joins, and time-windowed aggregations across diverse workloads. To support these asynchronous processing patterns across their Go ecosystem, the Coban team developed a managed, NoOps event sourcing and stream processing platform. The architecture packages stateless processing pipelines as Kubernetes deployments on AWS, polling Kafka event logs and using ScyllaDB as a shared metastore for stateful needs like deduplication and windowing. Stream processing pods combine ingestion triggers, a worker pool runtime, and user-provided domain logic plugins with customizable failure handling. This infrastructure scales to handle over 300 billion events weekly while maintaining workload isolation and elastic autoscaling.
Using Grab’s Trust Counter Service to Detect Fraud Successfully
Grab's Trust Platform team built the Counter service to detect fraud across business verticals like transportation, food, and payments. The platform replaces manual, multi-week engineering cycles with a self-service UI where data analysts can define and experiment with counters independently. Operating on an asynchronous ingestion and synchronous transaction model, the architecture evaluates incoming stream data, enriches it via internal services, and persists aggregated signals to ScyllaDB through Grab-Stats. A multi-bucket strategy partitions queries into fifteen-minute, hourly, and daily granularities to maintain low-latency query aggregations across wide time ranges under strict SLAs.
Data First, SLA Always
Grab's Data Engineering team transitioned from periodic batch ETL ingestion to a real-time change data capture architecture called Trailblazer after dataset sizes exceeded the petabyte mark. The previous system caused severe JDBC timeouts and heavy CPU loads when executing chunked or full-scan queries on unindexed upstream MySQL tables. To solve this, MySQL binary logs are captured via Debezium on Kafka Connect, buffered in Kafka, and ingested into a data lake using Spark Structured Streaming. Checkpoints are decoupled from local storage and persisted in a Redis cluster to simplify ingestion offset overrides and handle ephemeral compute clusters. The system incorporates extensive health monitoring across Airflow, Datadog, and custom services to maintain stream liveliness and avoid Kafka retention breaches.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.