AG-UI as the Agent–UI Contract Layer
AG-UI is a typed event contract between an agent backend and a frontend, turning streamed agent state into UI. This diagram maps the layers and shows where human review plugs in.
Loading…
Topic
Streaming
123 posts about Streaming. Every summary links to the original.
- 123 posts
- 18 companies
- 7 explainers
See the idea
Streaming explainers
Streaming ASR vs. Batch ASR: The Buffer-Latency Trade-off
Streaming and batch ASR pipelines compared: how input buffer length, speaker diarization, and where audio is processed trade latency against transcript quality and privacy.
When the Pipeline Pushes Back
Backpressure is a feedback loop that slows work when downstream capacity or health deteriorates. Compare its gains, costs, and signals for choosing a control point.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
How We Simplified Our Data Ingestion & Transformation Process
Grab evolved its real-time data ingestion pipeline after an initial architecture built on Spark Streaming and Python encountered operational complexity, node failures, and data loss from S3 eventual consistency. Because the streaming workload primarily handled event partitioning and ORC file generation, the team consolidated these tasks directly into an existing Golang processing service. They implemented sharded concurrent maps for high-throughput partitioning and optimized heap allocations to resolve memory bottlenecks. This refactor removed intermediate Avro conversions and intermediate storage hops. The simplified Go pipeline eliminated data loss and reduced processing lag from up to 13 minutes down to approximately 1 minute.
Querying Big Data in Real-time with Presto & Grab's TalariaDB
Grab developed TalariaDB to support real-time SQL querying over high-velocity event streams while maintaining predictable sub-second latencies and low infrastructure costs. The distributed time-series store retains only the most recent hour of data and integrates directly with Presto via its PrestoThriftService interface. Internally, TalariaDB uses the Go-based Badger key-value store to maintain an in-memory key index of metric names and timestamps while mapping columnar event payloads directly to disk. Ingestion occurs by processing pre-partitioned event batches written to Amazon S3 via SQS notifications. By combining a zero-copy decoder with parallel split evaluation across gossiping cluster nodes, the architecture scales horizontally while serving millions of events per second.
Mohammed Ridwanul Islam: How Mentorship, the T Model and a Pen Are the Keys to His Success
Mohammed Ridwanul Islam's career story connects his upbringing in Noakhali, Bangladesh, varied work and music experience, and interest in engineering with his current role on Shopify's Eventscale team. He describes how trying to build a game led him to write C# and later study Electrical Engineering at the University of Waterloo, where an internship became a full-time position. Eventscale develops tools, libraries, and infrastructure for collecting event-oriented streaming data across web, backend, and mobile platforms, supporting analytics and operational needs. Mentorship during a difficult team project taught him project management concepts, while habits, VIM, Java, and writing designs on paper shape his working practices. His advice is to build broad competence alongside deep expertise, and to prioritize problem-solving, values, social impact, and continued growth.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.