# How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…

[Netflix](https://yomu.fyi/company/netflix) · Netflix Technology Blog · Aug 7, 2026

**Type:** Problem & solution

## Summary

Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.

## Context

Netflix required a fast, flexible serving layer for its Real-Time Distributed Graph (RDG) to deliver sub-100ms query responses over billions of evolving nodes and edges. The system needed to support diverse workloads, balancing shallow, wide queries that cause massive I/O fan-out with deep, narrow multi-hop queries where sequential network hops create significant latency overhead.

## Approach / What changed

The team designed the query layer around breadth-first graph traversal, batching network lookups across frontier levels rather than following sequential depth-first paths. They implemented an async-first execution pipeline across small dedicated thread pools of 16 to 24 threads total, integrated selective caching via EVCache with TTLs aligned to data volatility, and introduced a layered override system for early filtering and per-hop limits.

## Takeaways

- Asynchronous composition enabled the serving layer to handle thousands of concurrent requests across only 16 to 24 threads without blocking on I/O, dramatically lowering infrastructure costs.
- Selective caching with EVCache matched TTLs to property volatility and skipped near-expiry nodes, achieving a 70 to 80 percent hit rate and reducing storage calls by 3 to 4 times.
- Breadth-first traversal batches lookups across frontier levels, replacing sequential network call chains with bounded parallel rounds to maintain low multi-hop query latency.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Caching](https://yomu.fyi/topic/caching), [gRPC](https://yomu.fyi/topic/grpc), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://netflixtechblog.com/how-and-why-netflix-built-a-real-time-distributed-graph-part-3-querying-the-graph-with-grpc-0f3468349607)