The Adventures of Rendezvous in Heroku’s New Architecture

Summary

Heroku’s migration of Rendezvous, the bidirectional proxy that connects Heroku Run sessions to one-off dynos, from EC2 Classic to AWS VPCs caused one customer’s app to hit H12 timeouts. The move replaced a single Elastic IP ingress and process-level TLS termination with a Network Load Balancer, whose larger TLS frames exposed a blocking flaw in Rendezvous’s single-threaded Ruby channel loop. That loop used IO.select on the TCP socket and dyno terminal, but OpenSSL::SSL::SSLSocket could retain unread buffered data after a partial read; IO.select saw no TCP bytes and blocked beyond the Heroku Router’s 30-second limit. Engineers reproduced the behavior across two languages, instrumented the instance agent, then fixed the channel with separate threads and corrected socket-reading and IO.select handling. After staging tests, the customer moved back to VPC Rendezvous and confirmed the issue was resolved.

Takeaways

1. 1

   Network Load Balancers provided 5–7x more throughput than the prior EIP approach in Heroku’s internal tests, but their larger TLS frames exposed the existing channel-loop bug.

2. 2

   IO.select monitored the underlying TCP socket while OpenSSL::SSL::SSLSocket could still hold unread data in its own buffer, allowing the loop to block despite readable application data.

3. 3

   The deployed fix used one thread for each connection side and changed socket-read handling; staging validation preceded the customer’s return to the VPC-based Rendezvous path.