---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Data-Centric Machine Learning: Building Shopify Inbox’s Message Classification Model](https://yomu.fyi/post/data-centric-machine-learning-building-shopify-inbox-s-message-classif.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 21, 2022
- Type: Problem & solution

Shopify Inbox needed to classify millions of monthly customer messages so merchants could prioritize responses and guide product decisions. The team adopted a data-centric approach, improving the dataset rather than primarily changing model architectures. An initial prototype used Universal Sentence Encoder embeddings, UMAP and HDBScan clustering, semi-supervised labeling, and a neural classifier, but reached about 70% accuracy with 35% high-confidence coverage. For version 2.0, the team created a more explicit taxonomy, used human-labeled data and fine-tuned DistilBERT, while addressing privacy and class imbalance through data handling and targeted annotation. In production, version 2.0 achieved about 90% accuracy and 80% high-confidence coverage, although the taxonomy and training data require ongoing updates as conversation trends change.


### [Implementing "seen by" functionality with Postgres](https://yomu.fyi/post/implementing-seen-by-functionality-with-postgres.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Victor
- Published: Jul 18, 2022
- Type: Benchmark

Tracking unique post views within Postgres presents trade-offs between counter accuracy, row bloat, and concurrent write performance. To evaluate options under an architecture constraint preventing external dependencies, a benchmark suite generated synthetic users, skewed post distributions, and replayed view actions. The benchmark tested four implementations: a naive counter column, an hstore key-value approach, an association table, and HyperLogLog (HLL). Benchmark results demonstrated that simple-hstore achieved the lowest average latency among deduplicating approaches at 2.15 milliseconds, closely followed by HLL at 2.16 milliseconds. Despite hstore's raw performance in prototypes, HLL is recommended for production because it circumvents row bloat as view counts grow while avoiding expensive row counts.


### [Spin Infrastructure Adventures: Containers, Systemd, and CGroups](https://yomu.fyi/post/spin-infrastructure-adventures-containers-systemd-and-cgroups.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 15, 2022
- Type: Incident / postmortem

Spin’s infrastructure team investigated instance instability after Kubernetes nodes experienced frequent failures, including roughly five nodes per day, and instances showed memory use that could exceed their assigned limits. Spin runs instances as Kubernetes pods, with systemd inside each container managing initialization and processes, and the team compared resource behavior across kubectl exec and Spin shell sessions. Stress tests showed that processes started through kubectl exec entered the kubepods cgroup hierarchy, while processes launched through Spin shell entered a hierarchy with an effectively unlimited memory limit, causing usage to appear on the node rather than the pod. Experiments comparing containerd, Docker, and Podman identified systemd cgroup delegation as the relevant difference: Podman isolated systemd and its child processes correctly, while the production setup used cgroupfs. The team identified switching COS to the systemd cgroup driver as the forthcoming fix, while noting that the bind-mounted cgroup filesystem enabled the escape.


### [Typeahead Search at Nextdoor](https://yomu.fyi/post/typeahead-search-at-nextdoor.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Jerry Tian
- Published: Jul 6, 2022
- Type: Problem & solution

Nextdoor built a proximity-based autocomplete service to power typeahead search and mention features across its hyperlocal platform for hundreds of millions of entities, including users and businesses. The system shards geographic data using Uber's open-source H3 geohashing library and stores prefix indexes in memory using Redis sorted sets. By adopting a Command Query Responsibility Segregation architecture, ingestion writes are processed on Redis primary nodes and replicated to read-only search nodes with under 10 milliseconds of replication lag. Dedicated APIs handle indexing, typeahead lookups, and ranking before returning hydrated results. Operating since August 2021, the service processes hundreds of millions of monthly typeahead queries while maintaining a P95 search latency below 30 milliseconds.


### [Shopify and Open Source: A Mutually Beneficial Relationship](https://yomu.fyi/post/shopify-and-open-source-a-mutually-beneficial-relationship.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 6, 2022
- Type: Explainer

Shopify describes its relationship with Ruby, Rails, and the open-source community as mutually beneficial, linking the company’s success to its Rails foundation and its engineering culture to the Rails Doctrine. It frames a 100-year company ambition around making Ruby and Rails viable for large teams, mature codebases, millions of lines of code, and billions of web requests through production performance, architecture guidance, upgrades, tooling, and selective typing. To support that goal, Shopify contributes projects and initiatives spanning developer tooling, gradual typing, VS Code support, automated upgrades, YJIT, Ruby performance work, TruffleRuby collaboration, $500,000 in research funding, Bundler and RubyGems, and RubyGems.org security. The post argues that company participation improves engineering skills, application longevity, and influence over foundational software, while describing contributors who range from occasional participants to full-time maintainers.


### [Accelerate Large Model Training using DeepSpeed](https://yomu.fyi/post/accelerate-large-model-training-using-deepspeed.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sourab Mangrulkar, Sylvain Gugger
- Published: Jun 28, 2022
- Type: Tutorial

Training large models on hardware with limited GPU memory frequently causes out-of-memory errors when using standard Distributed Data Parallel. To solve this bottleneck, Hugging Face Accelerate integrates DeepSpeed ZeRO data parallelism to shard optimizer states, gradients, and model parameters across workers. For a 900-million-parameter DeBERTa model, ZeRO Stage 2 increased the maximum per-device batch size from eight to forty while achieving a 3.5-fold training speedup over DDP without degrading accuracy or F1 score. Advanced configurations allow sequence-to-sequence chatbot finetuning and full ZeRO Stage 3 CPU offloading for models like the 1.5-billion-parameter GPT-XL, enabling batch size 16 training where DDP fails. Accelerate enables these memory optimizations through simple configuration files and the accelerate launch command with minimal or zero code modifications.


### [How We Built Hydrogen: A React Framework for Building Custom Storefronts](https://yomu.fyi/post/how-we-built-hydrogen-a-react-framework-for-building-custom-storefront.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 22, 2022
- Type: Explainer

Hydrogen is a React framework for building custom Shopify storefronts, created to help merchants use the Storefront API without giving up capabilities associated with Liquid storefronts. Shopify built it around React Server Components and React 18’s streaming server-side rendering, using component-level data fetching, a preload cache to parallelize nested queries, and experimental loggers for query waterfalls and unused GraphQL properties. The framework evolved through more than 1,200 pull requests and 3,000 commits, with rigid components removed after feedback and nearly all exported GraphQL fragments eliminated to make queries more discoverable. The post reports that some demo store routes cut load time in half and that Hydrogen is used in production at Allbirds, Shopify Supply, Shopify Hardware, and Shopify, while future plans include a new router and incremental adoption from Liquid storefronts.


### [Intel and Hugging Face Partner to Democratize Machine Learning Hardware Acceleration](https://yomu.fyi/post/intel-and-hugging-face-partner-to-democratize-machine-learning-hardwar.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Simon
- Published: Jun 15, 2022
- Type: Announcement

Intel has joined Hugging Face's Hardware Partner Program to accelerate Transformer training, fine-tuning, and inference on Intel platforms. Large Transformer models introduce latency bottlenecks in production workloads like search and chatbots, where hardware-level optimization typically requires tedious trial and error. To streamline model optimization, the collaboration introduces Optimum Intel, an open-source library integrating the Intel Neural Compressor for automated quantization, pruning, and distillation. A demonstration applies post-training dynamic quantization to a fine-tuned DistilBERT classification model using a CPU-only PyTorch setup. The quantized model converted 38 Linear and 2 Embedding operators to 8-bit integers, reducing evaluation duration by 1.34x while keeping the accuracy drop within a 5% threshold.


### [How We Built Shopify Party](https://yomu.fyi/post/how-we-built-shopify-party.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 14, 2022
- Type: Explainer

Shopify Party is a browser-based internal tool designed to make virtual hangouts more social and reduce video fatigue after Shopify moved to remote work. Its 3D spaces favor non-prescriptive play, letting participants explore and manipulate objects, join or leave at any point, and use a small control set rather than follow prescribed rules or win conditions. The initial version was built in three months with Unity 2020 and the Universal Render Pipeline, exported to WebGL, while Normcore synchronizes game state, provides VoIP and networked physics, and uses room time to keep autonomous objects aligned across clients. Performance work included third-party assets, GPU instancing for repeated models, simpler materials, and adjustable render resolution. The launch exceeded expectations, with hundreds of games played each week, and the tool is positioned as an option alongside video conferencing rather than a replacement.


### [The Journey to Cloud Development: How Shopify Went All-in on Spin](https://yomu.fyi/post/the-journey-to-cloud-development-how-shopify-went-all-in-on-spin.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 9, 2022
- Type: Problem & solution

Shopify’s development environment, the Ruby-based dev tool and xhyve-backed railgun VM, no longer handled the growing number of repositories, services, integrations, and resource demands around its monoliths. Developers needed more services, faced sluggish code-build-test loops, and encountered costly state changes when tophatting collaborators’ work. The Environments team concluded that development environments needed cloud-like elasticity, then tested an automatically configured local Kubernetes cluster and easy creation of Google Compute Engine VMs; the source also describes Isospin and Spin as the resulting direction. Local Kubernetes did not gain broad organic traction, while the current Isospin implementation is described as the team’s best iteration for developers working on majestic monoliths. The team plans to evolve it into an application-development library built on dev, add dependency inference, and address the tension between persistent environments and Kubernetes pod relocation.


### [Supabase Beta May 2022](https://yomu.fyi/post/supabase-beta-may-2022.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Ant Wilson
- Published: Jun 1, 2022
- Type: Announcement

Supabase announced several platform updates and performance improvements across its database, hosting, and developer tooling ecosystem for May 2022. Developers managing preview deployments on Jamstack platforms like Vercel and Netlify can now configure wildcard redirect domains such as \*.mydomain.com/welcome. For serverless workflows, Edge Functions support a -no-verify-jwt CLI deployment flag to facilitate direct invocation via external webhooks without mandatory token validation. In addition, Supabase exposed a Prometheus-compatible metrics endpoint to all users for real-time project monitoring and alerting. Database security policies also gained a dedicated Target roles configuration, replacing older role check expressions and cutting query execution time from 46 seconds to 7 milliseconds in one customer deployment.


### [To Thread or Not to Thread: An In-Depth Look at Ruby’s Execution Models](https://yomu.fyi/post/to-thread-or-not-to-thread-an-in-depth-look-at-ruby-s-execution-models.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 31, 2022
- Type: Explainer

Threaded servers such as Puma and thread-based Sidekiq are popular Ruby deployment choices because threads can increase throughput without a proportional increase in memory. The post compares process and thread models using a memory model that separates static application memory from per-request processing memory, then refines it with Copy on Write (CoW), where forked processes share pages until writes force copies. It explains why RSS can double-count shared pages, recommends PSS and Linux’s /proc/$PID/smaps\_rollup for more meaningful measurements, and gives a production example in which about two-thirds of static memory is shared. The conclusion is metric-dependent: preload applications to improve CoW, prefer fewer larger containers when appropriate, consider Unicorn for reliable request timeouts, and start web workloads at two threads unless substantial I/O wait justifies more; job processors may benefit from higher counts.


### [Graphcore and Hugging Face Launch New Lineup of IPU-Ready Transformers](https://yomu.fyi/post/graphcore-and-hugging-face-launch-new-lineup-of-ipu-ready-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sally Doherty
- Published: May 26, 2022
- Type: Announcement

Graphcore and Hugging Face expanded the range of machine learning modalities and tasks available in Hugging Face Optimum. Developers can now access ten transformer models optimized for Graphcore IPUs across natural language processing, speech, and computer vision. The available architectures include BERT, ViT, GPT-2, RoBERTa, DeBERTa, BART, LXMERT, T5, HuBERT, and Wav2Vec2, complete with IPU configuration files and ready-to-use pre-trained or fine-tuned weights. The integration supports the Bow IPU processor, which uses 3D Wafer-on-Wafer stacking to achieve up to 350 teraFLOPS of AI compute. Optimum also integrates with the Poplar SDK 2.5, enabling compatibility with frameworks such as PyTorch, TensorFlow, Docker, and Kubernetes.


### [Lessons Learned From Running Apache Airflow at Scale](https://yomu.fyi/post/lessons-learned-from-running-apache-airflow-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 23, 2022
- Type: Explainer

Shopify describes operating Apache Airflow 2.2 on Kubernetes with the Celery executor and MySQL 8 for data extraction, machine-learning training, Iceberg maintenance, and DBT modeling. In its largest environment, Shopify runs over 10,000 DAGs, averages over 400 tasks running at a given moment, and executes over 150,000 runs per day, while facing slow file access, metadata growth, ownership tracking, DAG-author control, traffic bursts, and resource contention. The authors describe replacing per-pod GCSFuse access with an in-cluster NFS volume synchronized to GCS, retaining GCS as the user-facing source of truth; they also use 28-day metadata retention, a YAML manifest, DAG policies, and scheduling controls including pools, priority weights, and Celery queues. These measures are presented as ways to improve file performance, limit database growth, identify owners, enforce constraints, and reduce workload interference, though the retention policy does not support durable history-dependent features such as long-running backfills.


### [Shopify Invests in Research for Ruby at Scale](https://yomu.fyi/post/shopify-invests-in-research-for-ruby-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 16, 2022
- Type: Announcement

Shopify says it has invested nearly half a million dollars over the past year in academic research directed toward Ruby and the needs of its community. The work is intended to make programming-language developments account for Ruby’s metaprogramming, loose C extension API, and large-scale workloads with frequent redeployments, rather than assuming long virtual-machine warmup periods. Collaborators include Laurence Tratt, who is investigating hardware meta-tracing and basic-block stitching to generate a just-in-time compiler from the Ruby interpreter; Steve Blackburn, whose group is putting the MMTk garbage-collection toolkit into Ruby; and Stefan Marr, who is studying interpreter speed and startup and warmup time. The program also connects these researchers with Shopify’s Ruby engineers and complements internal work on YJIT and TruffleRuby, with more research and academic papers planned.


### [Gradio 3.0 is Out!](https://yomu.fyi/post/gradio-3-0-is-out.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Abubakar Abid
- Published: May 16, 2022
- Type: Announcement

Gradio 3.0 introduces a ground-up redesign of the Gradio library to improve machine learning demo development and browser performance. The frontend has been rebuilt using modern technologies such as Svelte, resulting in smaller payloads, faster page load times, and cleaner visual designs suitable for embedding in blog posts. Component updates include revamping the Dataframe component to support CSV drag-and-drop, introducing new UI elements like Gallery, and adding the TabbedInterface class to group demos under distinct tabs. Developers also gain access to Gradio Blocks, a low-level Python API enabling custom layouts, multi-step interfaces, dynamic component property modifications, and complex data flows. A competition called the Gradio Blocks Party has been organized through the end of May to encourage community adoption.


### [Accelerated Inference with Optimum and Transformers Pipelines](https://yomu.fyi/post/accelerated-inference-with-optimum-and-transformers-pipelines.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: May 10, 2022
- Type: Announcement

Transformer-based models are moving into production for large-scale workloads, but default implementations remain slow and resource-intensive compared to traditional machine learning algorithms. Hugging Face Optimum addresses this issue by integrating performance optimization tools and accelerated runtimes like ONNX Runtime directly into standard Transformers pipelines. Developers replace standard model classes with Optimum equivalents to convert models to ONNX, run graph optimizations such as operator fusion, and apply quantization. In an evaluation on an AWS EC2 m5.xlarge CPU instance, optimizing and quantizing a RoBERTa question-answering model cut average latency from 117.61 ms to 64.94 ms while retaining 99.61 percent accuracy. Known limitations include a 2GB remote model size limit, lack of seq2seq support, omitted past key values in causal language models, and no local caching for optimized ONNX models.


### [Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel](https://yomu.fyi/post/accelerate-large-model-training-using-pytorch-fully-sharded-data-paral.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sourab Mangrulkar, Sylvain Gugger
- Published: May 2, 2022
- Type: Benchmark

Increasing parameter counts in modern machine learning models make loading and training them prohibitive on standard hardware. While Distributed Data Parallel replicates the entire model across GPUs, PyTorch Fully Sharded Data Parallel shards optimizer states, gradients, and parameters across data-parallel workers. Benchmarks on causal language modeling using GPT-2 Large and XL across Titan RTX GPUs demonstrate that FSDP increases allowable batch sizes and prevents out-of-memory errors encountered in standard DDP. Integrating FSDP through Hugging Face Accelerate enables these capabilities alongside CPU offloading to run models exceeding GPU memory limits. However, Accelerate requires preparing models before creating optimizers when using multiple models and notes that mixed precision support with FSDP currently has transformer compatibility issues.


### [Getting Started with Transformers on Habana Gaudi](https://yomu.fyi/post/getting-started-with-transformers-on-habana-gaudi.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Simon
- Published: Apr 26, 2022
- Type: Tutorial

Habana Labs and Hugging Face partnered to accelerate Transformer model training using Habana Gaudi accelerators, which offer up to 40% better price performance than recent GPU-based Amazon EC2 instances. Setting up the environment requires launching an Amazon EC2 dl1.24xlarge instance using the Habana Deep Learning Base AMI on Ubuntu 20.04. Inside an interactive PyTorch Docker container configured for Habana runtimes, users install the Optimum Habana package from source along with necessary dependencies. Fine-tuning the bert-large-uncased-whole-word-masking model on the GLUE MRPC task completes in 2 minutes and 12 seconds with an F1 score of 0.9181. Utilizing EC2 Spot Instances reduces the hourly compute cost by 70%, from $13.11 down to $3.93.


### [Code Ranges: A Deeper Look at Ruby Strings](https://yomu.fyi/post/code-ranges-a-deeper-look-at-ruby-strings.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 14, 2022
- Type: Explainer

Ruby's string model supports mutable and immutable operations, per-object or per-file freezing, and more than 100 encodings, while permitting byte sequences invalid for their attached encoding. This flexibility helps legacy and unusual platforms but makes encoding checks, validity scans, and character-boundary handling part of many operations. To reduce repeated O(n) validation scans, Ruby implementations cache each string's state in a four-valued code range: unknown, 7bit, valid, or broken. The post explains how these values map to public methods, how MRI stores them in two header bits, and how JRuby and TruffleRuby represent them differently. It also describes code ranges as inputs to fast and slow paths and as useful but fallible cache information, especially for native extensions and string-heavy metaprogramming.


[Newer posts](https://yomu.fyi/topic/performance/page/33.md) · [Older posts](https://yomu.fyi/topic/performance/page/35.md)