ZGateway: Learnings from Putting a Proxy in Front of ZippyDB

Summary

ZGateway is a stateless proxy tier introduced to unify traffic from ZippyDB’s large, diverse client fleet and address the reliability limits of direct access. By terminating TLS, authorizing requests, applying admission control and shaping, and routing through regional tiers, it replaces a many-to-many connection mesh with bounded client-to-gateway and gateway-to-server hops. Its embedded thick C++ client batches and coalesces shard requests, while a shared pipeline supports pure-proxy and read-through-cache tiers; per-use-case metrics, traces, and quota usage provide operational visibility. The gateway handles more than 1 billion operations per second, carries about 40% of ZippyDB traffic, and adds about 6% computational overhead to an average use case, with traffic projected to exceed 60%; production work also consolidated transaction state handling across 100% of transaction traffic without a reliability regression.

Takeaways

1. 1

   In the direct model, database-host fan-in grows linearly with the client population; with ZGateway, fan-in is approximately regions multiplied by shard density per host, independent of client and database fleet size.

2. 2

   A model using 20 regions, 500,000 database hosts, 30,000 proxy hosts, 1,000,000 clients, and 50,000 shards per client estimates a 97–98% per-host connection reduction and roughly 19x fewer persistent connections overall.

3. 3

   Transaction state handling was consolidated behind a flag in nine phases, reaching 100% of transaction traffic without a reliability regression and keeping gateway behavior aligned with server-side transaction evolution.