Topic

Envoy

1 post about Envoy. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Modal
Modal's serverless Servers
Modal introduces Servers, a low-latency HTTP serving solution designed for applications like LLM inference where milliseconds matter. Servers reduce p50 latency from 39ms to 6ms compared to Modal Web Functions by removing control-plane lookups and queuing from the hot path. The architecture uses a stateless proxy (fprs) for domain association and load balancing, with Envoy for TLS termination and HTTP/2 multiplexing. Two design principles governed development: maximizing resource sharing while minimizing interference, and making no network calls on the request path. The system supports autoscaling, regionalized routing, proxy authentication, and mirroring for A/B testing and continual learning.

The best engineering reads, once a week.

Topic

1 post about Envoy. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Modal
Modal's serverless Servers
Modal introduces Servers, a low-latency HTTP serving solution designed for applications like LLM inference where milliseconds matter. Servers reduce p50 latency from 39ms to 6ms compared to Modal Web Functions by removing control-plane lookups and queuing from the hot path. The architecture uses a stateless proxy (fprs) for domain association and load balancing, with Envoy for TLS termination and HTTP/2 multiplexing. Two design principles governed development: maximizing resource sharing while minimizing interference, and making no network calls on the request path. The system supports autoscaling, regionalized routing, proxy authentication, and mirroring for A/B testing and continual learning.

The best engineering reads, once a week.