Building a data plane from scratch: Stripe’s own high-performance distributed proxy

Summary

Stripe replaced Envoy in its service mesh after reliability, connection-cardinality, rollout-risk, and customization problems became increasingly difficult at its scale. Mesh-proxy, a Go proxy, implements HTTP/1.1 and HTTP/2, connects directly to Stripe’s service registry, and supports retries, hedging, rate limiting, and dynamic priority-based routing. Unlike Envoy’s worker-per-core model, it uses one read and one write goroutine per connection and multiplexes requests over a shared connection, reducing connection counts from one per worker to one per host. Load tests and deployment results showed about half the CPU under high load and roughly 50% lower latency, equivalent to around 3,000 days of cumulative inter-service latency saved daily. Direct registry integration also reduced unnecessary xDS updates, while custom Ruby balancing added 15–20% headroom before bursts overwhelm machines; Stripe is now pursuing anomaly detection and smart routing.

Takeaways

1. 1

   Mesh-proxy reduced high-load CPU usage by about half and cut latency by roughly 50% compared with Stripe’s Envoy deployment.

2. 2

   Using one shared connection per host instead of one connection per worker reduced connection cardinality for fleetwide services and let them run reliably on smaller machines.

3. 3

   Custom Ruby traffic balancing provided 15–20% more headroom before bursts overwhelmed backend machines, while direct service-registry integration reduced unnecessary xDS updates.