---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Making the Supabase Dashboard Supa-fast](https://yomu.fyi/post/making-the-supabase-dashboard-supa-fast.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Inian Parameshwaran
- Published: Dec 13, 2020
- Type: Problem & solution

Adding features to the Supabase single-page application dashboard risked performance regressions due to increasing JavaScript bundle sizes. To establish a baseline, the engineering team introduced next-bundle-analyzer and tracked Real User Monitoring alongside Core Web Vitals using Sentry. Optimization efforts focused on pruning dependencies, which included swapping Moment.js for day-js, replacing Joi with ajv, reverting crypto-js to version 3.3.0, and moving the 388 KB zxcvbn password module to a backend API. The team also implemented Next.js dynamic imports for heavy components like spreadsheet parsing, removed legacy server-side props to unlock Automatic Static Optimization, and configured long cache headers on assets. These architectural adjustments brought Core Web Vitals within recommended thresholds and lowered client-side page transition payloads below 200 KB of JavaScript.


### [Simplify, Batch, and Cache: How We Optimized Server-side Storefront Rendering](https://yomu.fyi/post/simplify-batch-and-cache-how-we-optimized-server-side-storefront-rende.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 10, 2020
- Type: Explainer

The post explains how a new Ruby-based server-side Storefront Renderer reduced the time required to serve Shopify storefront requests. It combines MySQL multi-statement queries, handcrafted SQL, a thin data-mapping layer built from plain old Ruby objects, query book-keeping with eager- and lazy-loading, multiple LRU caching layers, and techniques for reducing memory allocations. For a product page, one database round trip can load the product, variants, images, shop, theme, and related resources; later requests can replay previously observed queries early, while less frequently used data remains lazy-loaded. The resulting renderer serves 75% of requests in under ~45ms, 90% in under ~230ms, and 99% in under ~900ms, with average response time nearly five times faster than the previous implementation.


### [Organizing 2000 Developers for BFCM in a Remote World](https://yomu.fyi/post/organizing-2000-developers-for-bfcm-in-a-remote-world.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 7, 2020
- Type: Problem & solution

Shopify organized a cross-company program to prepare its platform for Black Friday and Cyber Monday (BFCM), coordinating more than 2,500 people responsible for over 1,000 deploys across services. Engineering Program Managers absorbed administrative, communication, and reporting work so BFCM leads could focus on technical challenges, while departmental champions assigned preparation tasks. Preparation covered maintenance, game days, load tests, capacity increases, weekly tracking, and a Caution Tape bot requiring risk-to-value assessments for pull requests in critical areas. Deploys were slowed through smaller batches, longer canaries, and a planned weeklong lock while communication used Slack, GitHub, documentation, presentations, emails, and office hours. The effort coincided with $5.1+ billion in BFCM sales and load testing plus resource scaling paid off, but developer productivity declined amid inconsistent messaging; future plans called for a more balanced approach, automation, and reusable communication templates.


### [A World Rendered Beautifully: The Making of the BFCM 3D Data Visualization](https://yomu.fyi/post/a-world-rendered-beautifully-the-making-of-the-bfcm-3d-data-visualizat.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 4, 2020
- Type: Explainer

The 2020 Black Friday Cyber Monday Globe and a related Live View visualization had to represent orders from one million merchants within two months while remaining realistic and performant. The team used a reusable layer architecture with three.js, physically based rendering, 32-bit EXR environment maps, and custom materials and shaders for the Earth and carbon-offset bubble effects. To display thousands of orders, it replaced arc-heavy rendering with particle-based visuals built from Points, BufferGeometry attributes, and ShaderMaterial. An automatic optimizer monitors frame rate and, when it stays below 55 fps for more than two seconds, lowers canvas pixel ratio to 1x and reduces environment-map resolution, helping low-power devices return to 60 fps. The resulting implementation became an internal library for future globes.


### [Pummelling the Platform–Performance Testing Shopify](https://yomu.fyi/post/pummelling-the-platform-performance-testing-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 2, 2020
- Type: Explainer

Shopify prepares for high-traffic sales events such as BFCM by repeatedly simulating customer traffic on representative test shops, aiming to expose bottlenecks before merchants experience them. Its Platform Conditioning team supports service-owning teams with approachable performance-testing tools and processes, distinguishing load tests for known traffic levels from stress tests that push services toward failure. A Go load generator runs Lua flows through lightweight VMs, models realistic traffic shapes and costly endpoints, and can issue tens of millions of requests per minute or more. Tests are initiated through Slack using the internal ChatOps tool Spy, which returns links to dashboards, logs, and results. Cronograma provides formal experiments with hypotheses, observations, conclusions, alert detection, Slack timelines, and comparable metrics such as response times, 5xx errors, and requests per minute; the tools have led to performance improvements and give Shopify confidence in future sales events.


### [Postgres Views](https://yomu.fyi/post/postgres-views.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Nov 18, 2020
- Type: Explainer

Postgres views serve as query shortcuts that execute underlying SQL statements upon retrieval without generating new tables or persisting duplicate data. By encapsulating complex multi-table joins, standard views provide query consistency across applications, simplify repetitive calls, improve logical schema organization, and enhance security by restricting sensitive columns. In contrast, materialized views physically store query results on disk, dramatically reducing read latency for heavy queries spanning millions of rows. Because materialized views introduce the trade-off of stale data, administrators must periodically run the refresh command based on workload tolerances for use cases like analytics and internal dashboards. Materialized views should not substitute query optimization, as underlying query efficiency remains essential.


### [How to Introduce Composite Primary Keys in Rails](https://yomu.fyi/post/how-to-introduce-composite-primary-keys-in-rails.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 29, 2020
- Type: Tutorial

Shopify addressed inefficient data access in its multi-tenant Rails application, where auto-incrementing primary keys interleaved records from many shops even though queries usually targeted one shop. InnoDB stores rows in B+ trees and loads pages into a buffer pool, so the team changed orders and other suitable tables to composite primary keys such as \[shop\_id, order\_id\] while keeping an auto-incrementing id secondary key and configuring Active Record to use id. Supporting Ghostferry, migration tooling, and data extraction systems also had to accommodate the schema change, with unique secondary constraints avoided in some cases because LHM migrations could deadlock. On the most queried table, common queries improved 5–6x, median buffer pool reads fell from 1.8 to 1.2, and elapsed database time dropped roughly one hour per day per shard, while inserts became about 10x slower.


### [How to Do an In-depth Liquid Render Analysis with Theme Inspector](https://yomu.fyi/post/how-to-do-an-in-depth-liquid-render-analysis-with-theme-inspector.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 20, 2020
- Type: Tutorial

Shopify theme code can create server-side rendering bottlenecks that leave customers waiting on a blank screen, so the article explains how to analyze Liquid performance with Shopify Theme Inspector. It describes flame graphs as representations of code paths and execution time, then compares clean Liquid with heavy loops, nested loops, and mixed global Liquid scopes. Accessing product attributes raised a 10-item example from 13 ms to 162 ms, while 50 products took 800 ms; two- and three-level nested examples rendered in 55 ms and 72 ms. The guidance recommends examining total iteration time, sawtooth or hair-comb patterns, unnecessary attributes, pagination size, and AJAX alternatives. It proposes aiming for 200 ms and keeping total page rendering under 500 ms, while noting that this alone does not guarantee a good LCP score.


### [Optimally Scaling Kafka Consumer Applications](https://yomu.fyi/post/optimally-scaling-kafka-consumer-applications.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shubham Badkur
- Published: Oct 13, 2020
- Type: Problem & solution

Grab's Coban platform runs Golang-based stream processing pipelines on Kubernetes, servicing roughly 400 billion events weekly from Kafka. The initial Horizontal Pod Autoscaler setup caused resource waste and uneven load distribution across Kafka partitions during scale-in and scale-out events. To resolve this, Grab moved to a fixed pod count matching the topic's partition count and adopted Vertical Pod Autoscaling, reducing resource usage versus requests by approximately 45%. The team also introduced Kubernetes priority classes to segment latency-sensitive workloads onto On-Demand nodes and non-critical jobs onto Spot instances. Additionally, overprovisioning via low-priority placeholder pods managed by Cluster Proportional Autoscaler enabled rapid pod rescheduling and reduced deployment delays.


### [Images as Code: Representing Localized and Evolving Products on Marketing Pages](https://yomu.fyi/post/images-as-code-representing-localized-and-evolving-products-on-marketi.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 7, 2020
- Type: Problem & solution

Shopify needed product visuals on more than 35 localized domains to stay accurate as features, currencies, languages, and product UI changed. An SVG experiment made translated text difficult to manage because SVG lacks word wrapping, while export settings, effects such as Gaussian blur, and repeated translation hooks created maintenance and performance problems. The replacement represented interfaces in HTML and JavaScript through ScaleContentAsImage, which wraps fixed-width content, computes available space, applies a CSS transform scale, preserves layout with a spacer, absolutely positions the transformed element, and updates calculations on resize. The team concluded HTML/JavaScript was more maintainable and flexible for design, documentation, animations, and localization, while noting remaining work around lazy-loaded images, reusable-visual documentation, and future CSS Houdini possibilities.


### [Supabase Alpha September 2020](https://yomu.fyi/post/supabase-alpha-september-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Oct 3, 2020
- Type: Announcement

Seven months into development, Supabase announced a series of platform updates across authentication, database tooling, and client libraries. The release introduced OAuth logins supporting Bitbucket, GitHub, GitLab, and Google, alongside table cloning and one-click Postgres extension management. In the SQL editor, users can now save favorite queries and access locally stored query histories directly within the browser. The web dashboard adopted Next.js automatic static optimization for improved responsiveness, while postgrest-js migrated to TypeScript and an isomorphic gotrue-js TypeScript library was built for Netlify GoTrue integration. Supabase is prioritizing a transition from Alpha to Beta by tracking open-source tool performance in a dedicated benchmarks repository.


### [The Life-Changing Magic of Tidying Ruby Object Allocations](https://yomu.fyi/post/the-life-changing-magic-of-tidying-ruby-object-allocations.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Richard Schneeman
- Published: Sep 16, 2020
- Type: Explainer

The post presents object-allocation profiling as a way to find Ruby performance hotspots, including code inside libraries such as Active Record. It uses memory\_profiler and derailed\_benchmarks to group allocations by gem, file, and source location, then examines whether each allocation can be removed or avoided. A benchmark compares a direct maximum comparison with an array-based version: the array allocation makes the latter 2.04x slower in the reported run. One optimization for cache\_version reuses a raw database timestamp string when it is safe, checking its type, format, timezone, and provenance before deleting separators and padding PostgreSQL-truncated values. This change reduces allocation from 743,842 bytes and 6,626 objects to 702,955 bytes and 6,063 objects, and the post reports up to 1.12x faster; combined optimizations reach 1.23x overall.


### [Block Sparse Matrices for Smaller and Faster Language Models](https://yomu.fyi/post/block-sparse-matrices-for-smaller-and-faster-language-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: François Lagunas
- Published: Sep 10, 2020
- Type: Announcement

Full dense layers in neural networks are often unnecessarily large and can be pruned without sacrificing precision, but available sparse algebra tools and PyTorch native implementations lack efficiency. To resolve this issue, the pytorch\_block\_sparse extension introduces BlockSparseLinear as a drop-in replacement for standard linear layers alongside BlockSparseModelPatcher for on-the-fly model modifications. The library uses NVIDIA CUTLASS C++ CUDA templates derived from Yulhwa Kim's cutlass tilesparse proof of concept. Although baseline sparse operations remain roughly twice as slow as cuBLAS-optimized dense counterparts, performance scales with sparsity, making a 75% sparse matrix approximately two times faster than a dense layer while reducing memory consumption by four times. Future updates will target dynamic sparsity optimization and NVIDIA Ampere Tensor Core primitives.


### [Uncovering the Truth Behind Lua and Redis Data Consistency](https://yomu.fyi/post/uncovering-the-truth-behind-lua-and-redis-data-consistency.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Allen Wang
- Published: Sep 7, 2020
- Type: Incident / postmortem

Grab experienced replica CPU usage spikes following service deployments in their master/replica Redis cluster, which caused failovers to spike to 100% CPU. Investigation revealed that a post-deployment Lua monitor script executed separately on both nodes and relied on non-deterministic HGETALL key ordering. Redis encodes hash objects as either ziplists or hashtables, and restoring from an RDB snapshot initializes small hashes as ziplists even if the master previously converted them to hashtables. This encoding discrepancy caused key ordering to diverge, preventing secondary data from deleting correctly and bloating dataset sizes. Grab resolved the issue by sorting the outputs of HKEYS and HGETALL within the Lua script to guarantee deterministic execution across nodes.


### [Migrating Large TypeScript Codebases To Project References](https://yomu.fyi/post/migrating-large-typescript-codebases-to-project-references.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 29, 2020
- Type: Tutorial

Shopify’s merchant admin UI grew from a Ruby on Rails ERB-based front-end into a TypeScript, React, and GraphQL codebase, and its size eventually made VSCode’s TypeScript language tooling take roughly two to three minutes to initialize. The team investigated the bottleneck with TypeScript’s performance diagnostics, VSCode TSServer logs, and support from the TypeScript team, which recommended splitting the monolithic configuration with project references. They built the TypeTrack VSCode plugin to measure initialization time, then migrated dependency-graph leaf nodes such as packages and tests, creating project-level tsconfig.json files and references incrementally. The migration reduced initialization times for sampled files from about 155 seconds to 8–13 seconds. The post explains how to resolve missing module errors, respect project-reference restrictions, and continue upward through the dependency graph while distributing migration work across teams.


### [Making Time to Save You Time: How We Sped Up Time-Related Syscalls on Dynos](https://yomu.fyi/post/making-time-to-save-you-time-how-we-sped-up-time-related-syscalls-on-d.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Will Farrington
- Published: Jul 16, 2020
- Type: Problem & solution

A customer support ticket exposed poor performance in clock\_gettime(3) and gettimeofday(2) for an application making many timing calls through transaction tracing. Heroku's Runtime Infrastructure team found that Linux was selecting the Xen clocksource, whose Xen version did not support virtual dynamic shared object (vDSO) acceleration, forcing costly userspace-to-kernel context switches. The team evaluated available clocksources and selected tsc, which uses the Time Stamp Counter and supports vDSO; they assessed concerns about backwards clock drift, including older CPU inconsistencies and Xen VM freezing, against their newer Intel CPUs and EC2 environment. After vdsotest validation, the configuration was deployed across the Common Runtime and Private Spaces dyno fleet, and the initiating customer plus other customers observed performance improvements.


### [The Reformer - Pushing the limits of language modeling](https://yomu.fyi/post/the-reformer-pushing-the-limits-of-language-modeling.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Jul 3, 2020
- Type: Explainer

Standard transformer models hit memory bottlenecks on long sequence modeling tasks due to the quadratic asymptotic memory complexity of global self-attention and oversized positional embedding matrices. The Reformer architecture overcomes these constraints to train sequences of up to half a million tokens using under 8GB of RAM. It re-engineers transformer operations using local and Locality Sensitive Hashing self-attention, chunked feed forward layers, reversible residual layers, and axial positional encodings. In empirical benchmarks using google/reformer-crime-and-punishment, axial positional encodings reduce the model parameter count from over 136 million to approximately 2.58 million by factorizing the positional dimensions. This architectural change cuts inference memory consumption from 959 MB down to 447 MB for evaluated benchmark workloads.


### [A First Look at Reanimated 2](https://yomu.fyi/post/a-first-look-at-reanimated-2.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 2, 2020
- Type: Explainer

Reanimated 2 is presented as a new approach to building gestures and animations in React Native, replacing the declarative domain-specific language of Reanimated 1 with animation worklets. Worklets are JavaScript functions declared with the “worklet” directive that execute independently on the UI thread, can receive parameters, access JavaScript-thread constants, call other worklets synchronously, and invoke JavaScript-thread functions asynchronously. The API introduces five hooks for creating shared and derived values, handling gesture events, and assigning animated styles or properties; a drag example combines shared values, gesture context, and velocity-based decay. The article also describes a liquid-swipe example using bezier interpolation, gesture handling, and physics-based animation, and reports that the new design lowers the entry barrier, supports cases previously requiring the bridge, and improves initialization performance.


### [Spark Joy by Running Fewer Tests](https://yomu.fyi/post/spark-joy-by-running-fewer-tests.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 11, 2020
- Type: Problem & solution

Shopify’s monolithic Ruby repository has more than 150,000 tests, grows 20–30% annually, and takes 30–40 minutes to run across hundreds of Docker containers, making required full-suite CI costly and vulnerable to intermittent failures. The team built a dynamic-analysis test-selection system that logs method calls during each test, records files in each call graph, and maps changed files to relevant tests. Because Ruby, Rails metaprogramming, and non-Ruby files complicate tracing, the system adds Rails patches and fallback rules, runs extra tests when mappings lag, and executes the full suite asynchronously on every deploy. After two months and 8,360 merged commits, it achieved 99.94% failure recall, selected about 60% of tests, and reduced compute time by roughly 25%. Developers requested full-suite runs on fewer than 2% of pull requests.


### [How to Fix Slow Code in Ruby](https://yomu.fyi/post/how-to-fix-slow-code-in-ruby.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 8, 2020
- Type: Tutorial

Performance regressions can accumulate in a large monolithic Rails application, making it difficult to identify offending changes among thousands of daily commits. The post presents profiling and benchmarking as complementary practices: profiling locates runtime bottlenecks, while benchmarking compares code paths and validates fixes. It covers elapsed time, CPU versus wall time, object allocations, TracePoint and ObjectSpace, plus rbspy, stackprof, rack-mini-profiler, and App Profiler, which supports on-demand remote production profiling at Shopify. A flamegraph example showed garbage collection consuming about 35% of CPU time in a slow request, and the team inferred excessive Ruby object allocation; a Rails benchmark showed roughly 50x improvement from caching an order’s total price calculation. The discussion cautions against micro-optimizations whose gains do not justify code changes and recommends addressing larger performance issues first.


[Newer posts](https://yomu.fyi/topic/performance/page/38.md) · [Older posts](https://yomu.fyi/topic/performance/page/40.md)