Loading…
Topic
gRPC
18 posts about gRPC. Every summary links to the original.
- 18 posts
- 8 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Logpoints Walkthrough
Logpoints in IntelliJ IDEA provide a way to inspect a running Java process without suspending it, making them useful when println logging or regular breakpoints would be disruptive. The walkthrough uses a Dockerized gRPC client and server whose incorrect tenant discount is traced by attaching to the server and adding expressions at runtime, without rebuilding or redeploying. Progressive logging shows the tenant value as JetBrains while the expected value is jetbrains, and the correct-discount block is never entered; the supplied fix uses equalsIgnoreCase and returns 2,000 basis points. The article also demonstrates testing changes through logpoint side effects, extending gRPC deadlines selectively, logging in dependencies, and using the ij-debugger AI agent skill. IntelliJ IDEA 2026.2 adds faster placement and instrumentation that removes debugger-introduced overhead, although heavy expressions still execute in the VM.
Enabling Seamless Kafka Async Queuing with Consumer Proxy
Uber’s Kafka deployment grew from 1 million to 12 million messages per second, but using Kafka’s ordered, partition-bound consumer model for asynchronous queueing created partition-scalability limits and head-of-line blocking. A one-second RPC example would require 1,000 partitions to reach 1,000 events per second, while non-uniform latency and poison-pill messages could stall unrelated work. Consumer Proxy addresses the mismatch by fetching messages through Kafka’s binary protocol, pushing them individually over gRPC, collecting processing statuses, and committing offsets only when safe. It also provides features including out-of-order commits, dead-letter-queue operations, flow control, and circuit breaking while hiding Kafka consumer-group management from services. At Uber, Consumer Proxy became the primary async-queueing solution, with hundreds of services onboarded; adaptive flow control and Kafka cluster federation are planned next steps.
How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…
Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.
Agent platform (Part 1): How we help Grab build and run AI agents at scale
Grab’s AI support bot grew from an internal GPT-powered Level-0 service into the foundation for LLM-Kit, an internal framework now used by more than 500 services, alongside over 50 registered remote MCP servers and a company-wide LLM gateway handling billions of tokens monthly. The bot used a single-agent reasoning loop and a tool plane spanning Glean, Kibana, GitLab, Slack, and HTTP plugins, but scaling exposed missing evaluations, fragmented observability, provider-switching friction, and costly production plumbing. LLM-Kit addresses those problems with generated FastAPI services, LangGraph agent templates, OpenTelemetry tracing, configuration and secret-management integrations, evaluation routes, remote MCP tool discovery, and gRPC contracts, clients, health checks, and channel selection. The post concludes that framework concerns became platform concerns at 500 agents, leading to a gateway, remote MCP framework, and evals platform described as the series continues.
Grabette: an open system to record robot-manipulation data
Grabette is an open, low-cost system for recording robot-manipulation data without requiring a robot, lab, or teleoperation rig. It uses a handheld gripper with a wide fisheye observation camera, an RGBD tracking camera, an IMU, and gripper encoders to capture synchronized demonstrations and recover camera-local 6-DoF Cartesian pose plus gripper state. Users save episodes on a Raspberry Pi, then select them in a browser dashboard for upload, RTAB-MAP SLAM, trajectory verification, and conversion into a LeRobot dataset on the Hugging Face Hub. The release also includes a motorized Gripette gripper and an example that trains a Diffusion Policy on 200 demonstrations and evaluates it on an OpenArm 7-DoF arm via gRPC, demonstrating the recording-to-training workflow.
GitFarm: Git® as a Service for Large-Scale Monorepos
At Uber’s monorepo scale, traditional Git workflows create bottlenecks through multi-gigabyte clones, local checkouts, repeated synchronization, and independent fetch or push operations across automation systems. The post describes GitFarm, a Git as a Service platform exposed through a high-performance gRPC API that executes standard Git commands in secure, ephemeral sandboxes backed by pre-warmed repository and container pools. Each backend maintains a bare clone, receives push-based updates and periodic git fetch operations every five minutes, while clients can explicitly fetch when they need current state. The design provides a full checkout in under 500 milliseconds and reduced client-side resource utilization by over 80%, while avoiding local clones and long initial clone delays.
Introducing uFowarder: The Consumer Proxy for Kafka Async Queuing
Uber introduces uForwarder, an open-source Consumer Proxy for Kafka Async Queue workloads, after deploying a push-based system across more than 1,000 consumer services. The proxy fetches messages through Kafka’s binary protocol, sends them individually to consumer services over gRPC, collects status codes and processing results, and commits offsets when safe. Production challenges included head-of-line blocking from messages that fail before reaching handlers, growing hardware costs, external data sources that are not ready when events arrive, and requirements for production/non-production and zone isolation. The implemented solutions include context-aware routing through message and gRPC headers, active blocking detection and mitigation, hardware-efficiency improvements, and DelayProcessManager, which pauses only delayed partitions and guarantees at-least delay rather than exact timing. The project is open source, with planned work on consumer-lag handling and direct Protobuf support.
Ingesting the Milky Way: Petabyte-Scale with Zerobus Ingest
The post benchmarks Zerobus Ingest, a managed, serverless, push-based service that writes producer data directly to Delta tables governed by Unity Catalog, against a petabyte-scale telemetry workload. Using NASA’s NEOWISE dataset and Locust, the test modeled fan-in from 2,048 concurrent streams, using Protocol Buffer 2 data over approximately 24 hours. The design replaces static partition-based ordering with stream-connection ordering, allowing heuristic routing across pods, dynamic partitioning, and autoscaling while existing streams drain. It also uses zeroparser, a zero-copy protobuf decoder whose design relies on Rust’s lifetime system and supports dynamic descriptors at about 1 GB/s per CPU core. The test sustained 12 GB/s to one table, ingested 1.04 trillion rows, and reached 1 petabyte within 24 hours; Zerobus Ingest is generally available, with additional APIs on its roadmap.
Rethinking Distributed Systems for Serverless Performance and Reliability
Building serverless compute for Apache Spark requires more than warm machine pools or basic autoscaling because traditional deployments couple user applications to infrastructure, allowing contention and inefficiencies to undermine performance and reliability. The proposed architecture separates these concerns through Spark Connect’s client-server model over gRPC, a gateway that routes workloads using query size, cluster utilization, and latency profile, and an adaptive autoscaler that adjusts capacity horizontally and vertically. Spark Connect isolates user applications from drivers, while the gateway continually re-evaluates placement to reduce interference between workloads. The autoscaler offers Standard and Performance-Optimized modes and can respond to out-of-memory errors by restarting tasks on larger VMs without manual intervention. Reported outcomes include a 99.998% upgrade success rate across more than 4.5 billion workloads, 2–5x faster Unilever pipelines, and operational-cost reductions of 25%.
How we built Stripe Credits: A programmable, auditable way to pay your Stripe fees
Stripe Fee Credits replaces ad-hoc discounts and manual cash adjustments with a programmable, auditable virtual balance dedicated to eligible Stripe fees. The architecture has three layers: a standalone gRPC credits engine for metadata, policies, lifecycle, and audit history; logical financial accounts built on Money Movement and Storage; and fee-settlement orchestration that applies eligible credits before the primary balance. Logical accounts carry ledger balances without representing real cash, while platform checks restrict Originated Money Transfers to logical-to-logical movements and exclude these accounts from cash sweeps. Prepaid and Stripe-funded credits share the user experience but use separate funds flows: prepayments create deferred revenue and collect tax at drawdown, whereas Stripe-funded credits are booked as Stripe costs with taxes written off. The resulting infrastructure supports incentives, prepaid annual fees, and other programs across 26+ countries, with Dashboard balances and real-time auditability, while avoiding bespoke engineering for each commercial model.
Grab's service mesh evolution: From Consul to Istio
Grab describes migrating its service mesh from Consul and the Catcher fallback across a microservices ecosystem of more than 1,000 services spanning VMs, EKS clusters, AWS, and GCP. The existing setup exposed fleet-wide risk from a single Consul server issue, added fallback complexity, and lacked advanced traffic management, security controls, and robust multi-cluster support. After evaluating Application Load Balancer, Istio, AWS Lattice, and Linkerd for stability, performance, integration, and operational simplicity, the team selected Istio for its Kubernetes integration, flexible deployment models, and feature set. The design uses multiple control planes on dedicated Kubernetes clusters, active-active pairs, organization-based mesh boundaries, custom resource definitions, and cross-cluster endpoint discovery to support isolation, availability, routing, security, and observability. Migration began in Q4 2024 with GCP-to-AWS work alongside gRPC and HTTP traffic initiatives, using gradual shifting, rollback mechanisms, and performance monitoring while teams manage legacy systems and cross-mesh connectivity.
Counter Service: How we rewrote it in Rust
The Integrity Data Platform team rewrote Counter Service, a high-QPS Golang microservice that consumes streams, writes counts to Scylla, and exposes gRPC queries backed by Scylla and Redis. They selected it because BatchRead reached tens of thousands of QPS while remaining constrained in functionality, making infrastructure savings potentially worth the rewrite effort and Rust learning curve. Rather than translating code line by line, the team treated the existing service as a black box, preserved its interfaces and contracts, and rebuilt the application logic from scratch; it also assessed Rust ecosystem support for dependencies including tonic, Actix Web, Fred, and the Scylla driver. The rewrite delivered similar performance with 70% infrastructure savings, according to the abstract. The conclusion reports the service as 80% cheaper, endorses Rust for new services, and cautions that async Rust is challenging.
How Tinder Eased Development Pain With Ignis
Managing hundreds of microservices with disparate configurations forced Tinder engineers to deploy code to staging environments to verify every change, slowing development velocity. To shorten feedback loops, the team created Ignis, a Go-based CLI tool that lets engineers run an isolated service locally while proxying ingress and egress traffic through a multi-tenant development Kubernetes cluster. Ignis pairs with an API gateway for client testing and uses a custom Smart Router proxy to dynamically intercept downstream gRPC and HTTP requests via Kubernetes informers and header propagation. Since launching in 2019, the tool enabled Tinder to eliminate a dozen staging environments, decrease cloud spend, and accelerate onboarding, followed by a V2 release adding remote IDE support and monitoring.
Message Center - Redesigning the messaging experience on the Grab superapp
Grab redesigned its messaging infrastructure from GrabChat to Message Center to overcome two-party chat limitations and support complex superapp requirements like group conversations and varied user roles. The architecture separates core processing logic from message delivery by splitting the system into a backend processor and an independently scalable postman service. Communication relies on an in-house TCP gateway named Hermes that proxies client payloads via gRPC, alongside Apache Kafka streams and Amazon SQS delay queues. Custom client-server acknowledgements and a DynamoDB event store ensure reliable message delivery even across dropped mobile TCP connections and offline reconnects.
Serving Driver-partners Data at Scale Using Mirror Cache
Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.
Faster TensorFlow models in Hugging Face Transformers
Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.
Using Grab’s Trust Counter Service to Detect Fraud Successfully
Grab's Trust Platform team built the Counter service to detect fraud across business verticals like transportation, food, and payments. The platform replaces manual, multi-week engineering cycles with a self-service UI where data analysts can define and experiment with counters independently. Operating on an asynchronous ingestion and synchronous transaction model, the architecture evaluates incoming stream data, enriches it via internal services, and persists aggregated signals to ScyllaDB through Grab-Stats. A multi-bucket strategy partitions queries into fifteen-minute, hourly, and daily granularities to maintain low-latency query aggregations across wide time ranges under strict SLAs.
Mockers - Overcoming Testing Challenges at Grab
Grab operates over 250 microservices communicating over HTTP and gRPC, making shared staging environments costly, ambiguous in ownership, and fragile due to inconsistent data and uncoordinated deployments. To address these testing bottlenecks, Grab created Mockers, a Go SDK and CLI tool backed by a central monorepo of mock servers for local-box and CI testing. Mockers automatically generates HTTP and gRPC mock servers from Swagger specifications and protobuf files, returning configured network responses without internal business logic. By incorporating Grab's in-house chaos SDK middleware, Mockers also enables repeatable resiliency and contract testing locally without relying on code-level mocks. While Grab still mandates integration testing on distributed staging environments with live data, Mockers enables developers to detect complex defects and contract mismatches earlier.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.