Loading…
Browse by type
Benchmark reads
Benchmark posts on Yomu, summarised newest first.
- 167 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Benchmark posts
Turboku - The Art Of Faster Heroku Apps
Turboku is a migration path that automatically turns Heroku web dynos into Fly applications, addressing how existing Heroku apps can move to Fly’s globally distributed edge network. Fly routes requests to the closest datacenter through one global IP address, or two with IPv6, while TLS terminates closer to users and modern ECDSA certificates are automatic for Fly domains. A benchmark around the world used an existing application to compare Heroku and Fly connection stages, full requests, and cached requests. Fly reduced DNS, TCP, TLS, network, and full-request times, with the largest reported improvement appearing when caching was used: 180.16 ms versus Heroku’s 817.75 ms, a 77.91% reduction; uncached full requests fell 35.39%, while TTFB was 1.24% higher.
Bug Bounty Year in Review 2018
Shopify’s 2018 bug bounty review reports its fifth year and most successful program to date, with faster handling and higher payouts. The security team cut average triage time from four days in 2017 to 10 hours in 2018 by assigning one of its 14 Application Security members to HackerOne triage each week. At Montreal’s H1-514 event, over 40 hackers produced more than 50 valid reports, supported by early submissions, disclosed resolved reports, incentive bonuses, expanded scope, and infrastructure shell access. Excluding event statistics, average bounty rose to $1,790, total payouts reached $155,750, 60% of resolved reports received a bounty, and triage, bounty, and resolution times improved, although resolved reports and participating hackers declined and some low-severity reports exceeded the one-month target.
Shaping the Future of Payments in the Browser
Shopify tested whether the Payment Request API was ready for its merchants and buyers on supported browsers. The team defined checkout completion rate, time to purchase, drop-off, and qualitative checkout experience as success measures, then ran an A/B experiment with selected merchants and eligible buyers. The isolated Ruby app injected storefront JavaScript, checked eligibility and credit-card support, and routed buyers to either the regular checkout or a browser-provided payment sheet. The experiment recorded a 7% checkout conversion rate drop and a 60% drop at the payment sheet, although time to completion medians were promising. Shopify concluded Payment Request was not yet a good fit, while expecting greater familiarity, payment-sheet improvements, and third-party payment methods through Payment Handler API to help adoption.
Iterating Towards a More Scalable Ingress
Shopify’s infrastructure uses Kubernetes to serve millions of requests per minute, and its growth exposed ingress performance problems during frequent application deployments. After replacing Google Cloud Load Balancer Controller (glbc) with endpoint-aware ingress-nginx, the Edgescale team focused on reducing the NGINX reloads that caused latency increases. They added dynamic configuration: the controller POSTs endpoint lists to an NGINX HTTP endpoint, where lua-nginx-module updates an in-memory shared dictionary instead of regenerating the configuration file and reloading NGINX. Reload avoidance preserves keepalive connections and load-balancing state while reducing the resource effects of replacing worker processes. Tests found ingress-nginx outperformed glbc by multiple orders of magnitude at the 99.99th latency percentile or higher, while dynamic configuration produced its largest gains during deploys, especially beyond the 80th percentile.
Apache Kafka 0.10 Evaluating Performance in Distributed Systems
Testing Apache Kafka 0.10 revealed an apparent 33% throughput decline versus Kafka 0.9, prompting an investigation into whether the regression affected different plans, connection types, message sizes, and workload scales. Heroku’s benchmark system used multiple producer and consumer dynos, while the team applied Brendan Gregg’s USE method to examine resource utilization, saturation, and errors. The diagnosis was network-card saturation: Kafka 0.10 adds an 8-byte timestamp to every message, which at 1.5 million messages per second and replication factor 3 produced 288 Mbps of additional cluster traffic, causing packet drops and retransmits. Reproducing the effect by adding 8 bytes to Kafka 0.9 confirmed the explanation, and the post says most production workloads are unlikely to be affected, though clusters near network limits may be.
We Patched Ruby On Rails for a 10% Performance Bump
Rails response time improved by more than 10% in a patch that also reduced allocated objects by 29%, motivated by memory and performance problems observed in production Heroku applications. The work began with customer reports of R14 out-of-memory errors, swap usage, and suspected memory leaks, then used a customer app and reusable benchmarks to reproduce issues. The derailed benchmarks perf:objects task and memory_profiler exposed allocation hot spots, including arrays joined into strings and duplicated hashes; refactoring reduced unnecessary objects, while routing changes produced a 31% speed increase for constrained URL generation. The resulting savings required no application changes and were planned for the next Rails version, with Rails 5 pre-release testing mentioned.
Benchmarking Rack Middleware and Improved Speed
ActionDispatch::Static, enabled by config.serve_static_assets = true, checks incoming Rails requests to decide whether to serve a disk file or pass the request onward. To measure the middleware in isolation, the author loads it directly, wraps it around a no-op Proc, and uses Rack::MockRequest with benchmark-ips to compare requests against the no-op baseline. The original middleware achieved 15,891.2 iterations per second, versus 28,660.9 for the no-op, while the modified version reached 24,490.9 iterations per second. That change made non-asset middleware requests roughly 54% faster, bringing the gap to the no-op case to about 10%, although asset requests became slightly slower and the difference was not measurable. Across the full Rails application stack, the reported overall speedup was approximately 2.6%.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.