How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…

Summary

Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.

Takeaways

1. 1

   Asynchronous composition enabled the serving layer to handle thousands of concurrent requests across only 16 to 24 threads without blocking on I/O, dramatically lowering infrastructure costs.

2. 2

   Selective caching with EVCache matched TTLs to property volatility and skipped near-expiry nodes, achieving a 70 to 80 percent hit rate and reducing storage calls by 3 to 4 times.

3. 3

   Breadth-first traversal batches lookups across frontier levels, replacing sequential network call chains with bounded parallel rounds to maintain low multi-hop query latency.