How We Sped up SNI TLS Handshakes by 5x

Summary

Heroku’s SNI TLS termination work exposed severe handshake performance problems under benchmarks designed to avoid session resumption, caching, and keep-alive reuse. Initial tests made nodes slow and unresponsive 7–8x earlier than comparable HTTP tests, while perf top showed Erlang garbage collection and ETS hash operations consuming unexpectedly large amounts of CPU. Tracing linked repeated ets.select_delete calls to the centralized ssl_manager process, where ssl_pkix_db’s certificate cache path was still used even for DER certificates supplied directly in memory. Engineers patched the SSL library to bypass that cache wholesale. Each instance then handled three times the benchmark capacity, while SNI latencies fell from 100–120 milliseconds to 20–25 milliseconds for the reported app subset; the patch was later shipped in Erlang/OTP-19.1.

Takeaways

1. 1

   Handshake-focused benchmarks disabled session resumption, caching, and keep-alive reuse while using very small HTTP requests to make handshake cost dominate.

2. 2

   perf top and Erlang tracing identified ETS hash work and garbage collection around ssl_manager as the bottleneck, rather than cryptographic processing in libcrypto.so.

3. 3

   Bypassing the certificate cache tripled benchmark capacity and reduced reported SNI latency from 100–120 ms to 20–25 ms; the change shipped in Erlang/OTP-19.1.