---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Machine Learning Experts - Lewis Tunstall](https://yomu.fyi/post/machine-learning-experts-lewis-tunstall.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Britney Muller
- Published: Apr 13, 2022
- Type: Explainer

In this interview with Britney Muller, Hugging Face machine learning engineer Lewis Tunstall discusses his background, his book Natural Language Processing with Transformers, and tooling optimizations for deploying transformer models. Tunstall notes that large model parameter sizes often lead to high latency, making off-the-shelf transformers impractical for responsive applications like chatbots. To resolve this issue, he developed single-line export tooling within the transformers library to convert models into the ONNX format. This conversion enables engineers to run PyTorch models in TensorFlow or on specialized hardware, achieving faster latency and higher throughput. Tunstall also recounts co-authoring the O'Reilly book with Leandro von Werra and Thom Wolf, collaborating on the Hugging Face Course, and training a GPT-2 model named code parrot on Python code.


### [Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training](https://yomu.fyi/post/habana-labs-and-hugging-face-partner-to-accelerate-transformer-model-t.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Susan Lansing
- Published: Apr 12, 2022
- Type: Announcement

Training transformer models across computer vision, speech, and natural language processing tasks at scale often demands heavy compute resources, incurring significant time and financial expense. To address these bottlenecks, Habana Labs and Hugging Face partnered to integrate the SynapseAI software suite into the Hugging Face Optimum open-source library. This integration allows machine learning practitioners to accelerate transformer training workflows on Habana Gaudi processors using minimal code adjustments. Habana Gaudi hardware, featured in Amazon EC2 DL1 instances and Supermicro X12 servers, incorporates ten 100 Gigabit Ethernet ports per processor to scale from single units to thousands of chips. The combined hardware and software architecture supports TensorFlow and PyTorch while delivering price and performance metrics up to 40% lower than comparable training alternatives.


### [Supporting large campaigns at scale](https://yomu.fyi/post/supporting-large-campaigns-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jie Zhang
- Published: Apr 1, 2022
- Type: Problem & solution

Grab developed a batch job service within its Trident automation engine to execute multi-step marketing campaigns for millions of users simultaneously. The system replaces sequential, single-server execution with a distributed architecture powered by Apache Kafka, which distributes batches of 100 users across server clusters using hashed partition keys. To reduce network overhead and queries per second, downstream reward and messaging services introduced batch endpoints backed by bulk database queries, decreasing API latency by up to 85%. Grab further optimized performance by sharding Kafka topics by country and action type to prevent long-running reward tasks from blocking time-sensitive messaging workloads. Additionally, making terminal messaging calls asynchronous allows subsequent batch processing to proceed without waiting for message delivery confirmations.


### [Best-in-Class Developer Experience with Vite and Hydrogen](https://yomu.fyi/post/best-in-class-developer-experience-with-vite-and-hydrogen.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 31, 2022
- Type: Explainer

Hydrogen, Shopify’s framework for custom React storefronts, chooses Vite to improve developer experience while targeting performance for end users and evergreen browsers. Traditional bundling with tools such as Webpack can become complex and slow during development because each change may require processing more project files; Vite instead serves user code through native ES Modules, while pre-bundling and caching third-party dependencies. Vite combines an HMR development server with sensible defaults, ESBuild compilation, Rollup’s plugin system, CommonJS-to-ESM conversion, and SSR support, including transformations that differ between browser and server environments. Hydrogen integrates Vite through four plugins covering Hydrogen configuration, React Server Components, SSR and RSC middleware, and React fast refresh, enabling server components, streaming requests, and caching; the text concludes that Vite is a suitable choice, while acknowledging Webpack remains appropriate for some advanced use cases.


### [GraphQL is now available in Supabase](https://yomu.fyi/post/graphql-is-now-available-in-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Oliver Rice, David Thyresson
- Published: Mar 29, 2022
- Type: Announcement

Supabase announced the general availability of GraphQL support on its platform via pg\_graphql, an open-source PostgreSQL extension that reflects SQL schemas into GraphQL types, fields, and relationships. Accessible through SQL functions or HTTP endpoints, the extension enforces PostgreSQL row-level security and filters introspection schemas according to the SQL roles assigned to request JWTs. To optimize performance, pg\_graphql transpiles every GraphQL query into a single SQL query that aggregates results into JSON, avoiding the N+1 query problem and eliminating redundant join data payloads. Transpiled queries are cached as prepared statements to bypass planning overhead, and all generated mutations and queries support bulk operations. On Supabase Free Plan hardware, these optimizations achieve mean throughputs of 377.4 requests per second over the API and 656.2 queries per second over SQL connections.


### [How We Built the Add to Favorite Animation in Shop](https://yomu.fyi/post/how-we-built-the-add-to-favorite-animation-in-shop.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 25, 2022
- Type: Tutorial

The Shop app’s Add to Favorite interaction animates a product thumbnail into the Favorites tab bar heart after a buyer favorites a product. The implementation is based on a motion spec that defines animated properties, target values, durations, easing curves, and ordering, with the thumbnail sequence followed by the icon’s color and bouncing movement. In React Native, Reanimated uses one shared progress value for the whole sequence, and interpolation maps progress ranges to opacity, scale, and vertical translation, including clamping and an instant disappearance at step 3. A 50 ms pause is included between thumbnail scaling and its downward movement. The shared value also creates a match cut between the disappearing thumbnail and the icon; a setTimeout-based handoff was rejected because a busy JS thread could delay the transition, while withDelay would keep timing on the UI thread.


### [Accelerate BERT inference with Hugging Face Transformers and AWS Inferentia](https://yomu.fyi/post/accelerate-bert-inference-with-hugging-face-transformers-and-aws-infer.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Mar 16, 2022
- Type: Tutorial

Production deployments of BERT and Transformer architectures often face cost and latency challenges because these models are significantly larger and more computationally intensive than traditional algorithms. To optimize text classification workloads, developers can compile vanilla PyTorch models for AWS Inferentia using the AWS Neuron SDK and its tracing utilities. Because the Neuron SDK requires static tensor dimensions, the model is traced with fixed input lengths and packaged alongside a custom inference script configuring one Neuron Core per worker. Deploying the resulting artifacts to an Amazon SageMaker ml.inf1.xlarge endpoint yields an average latency of 5 to 6 milliseconds for a sequence length of 128 across 10,000 synchronous evaluation requests.


### [After the Refactor: A Path to Faster Rendering with Liquid-C](https://yomu.fyi/post/after-the-refactor-a-path-to-faster-rendering-with-liquid-c.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 10, 2022
- Type: Explainer

Liquid-C is an extension to the Liquid gem that parses, tokenizes, and renders templates, but some Liquid features, including If and For tags, still render in Liquid rather than Liquid-C. The work explains how Liquid-C compiles templates into bytecode for a customer VM and addresses the difficulty of moving instruction and constant pointers when control flow requires forward or backward jumps. It refactors instructions to store each constant’s index in the constants array, using a hash table to avoid duplicate constants and remove the constant pointer. The change reduced one example constants array from 157 objects to 104, but did not drastically improve theme rendering speed; it slightly increased memory usage and parsing time while preparing for GOTO or JMP operations and later If and For support.


### [Postgres Auditing in 150 lines of SQL](https://yomu.fyi/post/postgres-auditing-in-150-lines-of-sql.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Oliver Rice
- Published: Mar 8, 2022
- Type: Tutorial

Traditional PostgreSQL auditing methods duplicate source table structures, creating maintenance overhead when schemas change. To solve this, a compact SQL implementation stores insert, update, and delete events across multiple tables in a single audit table using JSONB columns. Query performance is maintained by using a BRIN index on naturally ordered insertion timestamps, a B-tree index on internal table OIDs, and UUIDv5 identifiers hashed from primary keys. Row-level PL/pgSQL triggers automatically populate the audit log and can be enabled or disabled dynamically with dedicated tracking functions. While trigger-based auditing introduces minimal overhead at rates below 1,000 writes per second, write-heavy workloads may benefit from logging changes outside SQL with tools like pgAudit.


### [7 Tips For Optimizing Apache Flink Applications](https://yomu.fyi/post/7-tips-for-optimizing-apache-flink-applications.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 8, 2022
- Type: Tutorial

Shopify presents lessons from operating large, stateful Apache Flink applications on Kubernetes with Google Kubernetes Engine, RocksDB, and checkpoints and savepoints stored in Google Cloud Storage. The guidance begins with profiling tools, including async-profiler, VisualVM, jemalloc with jeprof, and Eclipse Memory Analyzer, to investigate CPU, heap, native-memory, and leak-related problems. Disabling Kryo fallback exposed unsupported serializers such as Scala BigDecimal and Scala algebraic data types; replacing them with supported alternatives increased throughput by 20 percent. The post also recommends separate configuration profiles for high-throughput backfills and low-latency steady state, while describing a RocksDB native-memory failure in which disabling the block cache stopped repeated out-of-memory crashes without affecting application performance.


### [Test Budget: Time Constrained CI Feedback](https://yomu.fyi/post/test-budget-time-constrained-ci-feedback.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 7, 2022
- Type: Benchmark

Shopify’s core monolith contains more than 170,000 tests, and variable CI feedback times create delays and context switches as the test pool grows. The Test Infrastructure team evaluated a Test Budget approach that builds prioritized orders on top of deterministic test selection, then stops execution at a predetermined time to increase the chance of finding failures early while accepting some risk. Six criteria were compared—failure\_rate, avg\_duration, churn, coverage, complexity, and default/random—using Time to First Failure, APFD, and Convergence Index; historical results were stored and queried with Redis Sorted Sets. The analysis found failure\_rate had the best detection rate, while churn and complexity were inappropriate; running 60% of the suite detected 80% of failures in the mean case, and 70% of the test-selection suite detected 50% at the fifth percentile.


### [Adding the V8 CPU Profiler to v8go](https://yomu.fyi/post/adding-the-v8-cpu-profiler-to-v8go.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 4, 2022
- Type: Benchmark

v8go adds support for V8’s CPU Profiler, allowing Go users to measure JavaScript execution in a V8 isolate and inspect sampled top-down call trees with function and engine details. The API work aimed to stay idiomatic for Go, remain close to V8’s API, support future profiler features, and minimize Cgo overhead. The first design loaded profile data lazily, making separate Cgo calls for profile and node properties; the alternative built the complete graph in C++ before returning it to Go. Benchmarks showed lazy loading was faster for a small tree, averaging about 20 microseconds versus 25 for eager loading, while a larger tree reversed the result at roughly 90 versus 60 microseconds. The comparison demonstrates that Cgo costs depend on workload size and that performance assumptions require benchmarks using representative inputs.


### [RubyConf 2021: The Talks You Might Have Missed](https://yomu.fyi/post/rubyconf-2021-the-talks-you-might-have-missed.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 2, 2022
- Type: Announcement

RubyConf 2021: The Talks You Might Have Missed compiles talks presented by Shopify’s Ruby and Rails infrastructure engineers at the November conference in Denver, offering ways to revisit sessions for attendees who missed them. The program covers Ruby implementation topics including compiler history, Aaron Patterson’s pure-Ruby JIT compiler, YJIT’s incremental integration inside CRuby, parser generators, variable-width allocation, and Ruby memory layout. Other sessions address gradual Sorbet adoption, native extension compilation and security, pair programming, memoization performance, executable code as data, Ruby archaeology, fast CI, and Ractor-based parallel testing. The roundup reports early YJIT performance results, describes a CI environment with 2.8 million Rails monolith lines and 210,000 Ruby tests, and notes variable-width allocation passed Shopify’s Rails CI while serving more than 500 million requests in a week of limited production traffic.


### [React Server Components Best Practices You Can Use with Hydrogen](https://yomu.fyi/post/react-server-components-best-practices-you-can-use-with-hydrogen.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 4, 2022
- Type: Tutorial

React Server Components (RSC) introduced a paradigm shift for the team building Hydrogen, a React-based framework for custom storefronts. The post presents patterns developed through trial and error, beginning with shared components as a middle ground and reserving client components for interactivity, useState or useReducer, lifecycle logic, unsupported libraries, or browser APIs. Components should instead become server components when they contain secrets or proprietary logic, need filesystem or database access, fetch storefront API data, or do not execute on the client. The examples extract newsletter form controls and FAQ accordion behavior into client components, then rename the remaining wrappers as server components when their usage allows it; the FAQ server component is passed through a client component as children.


### [Rapid Development with Hydrogen: Building a Product Page](https://yomu.fyi/post/rapid-development-with-hydrogen-building-a-product-page.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 2, 2022
- Type: Tutorial

Updated for Hydrogen 0.26.0, the tutorial builds a simple product page for a custom Shopify storefront using Hydrogen’s React-based framework. Starting from the Hello World template on StackBlitz, it adds Tailwind CSS, creates a file-based dynamic product route, and queries product data through the Storefront API with the useShopQuery hook. The implementation then uses React Server Components alongside a client component and ProductOptionsProvider to track selected variants and options, update prices, and support a BuyNowButton checkout flow. A media gallery and additional styling complete the page, which uses products and media from the Hydrogen Preview store. The post concludes that Hydrogen’s components, hooks, and utilities reduce repetitive storefront work and speed development.


### [Cloud, Load, and Modular Code: What 2022 Looks Like for Shopify](https://yomu.fyi/post/cloud-load-and-modular-code-what-2022-looks-like-for-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 20, 2022
- Type: Announcement

Shopify’s 2021 BFCM four-day event exposed the scale its commerce platform must absorb: peak app-server traffic exceeded 32 million requests per minute, load balancers surpassed 34 million RPM, and a flash sale used more than 20% of total computing capacity. The post frames larger, spikier sales, mobile and social commerce, and expanding buying environments as enduring engineering challenges for supporting millions of merchants. Planned work includes evolving the large Rails codebase into modular components, moving storefront rendering out of the monolith, expanding Spin cloud development environments, and making in-house load testing part of product development. The storefront split and refactoring produced four-times-faster cache fill rates and five-times-faster page rendering, while YJIT ran 31% faster than interpreted CRuby and 26% faster than MJIT; Shopify also remains committed to Rails and React Native.


### [Case Study: Millisecond Latency using Hugging Face Infinity and modern CPUs](https://yomu.fyi/post/case-study-millisecond-latency-using-hugging-face-infinity-and-modern.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid, Jeff Boudier, Morgan Funtowicz
- Published: Jan 13, 2022
- Type: Benchmark

Deploying large Transformer models at scale often faces severe prediction latency bottlenecks, driving up infrastructure costs and limiting real-time production use cases. To address this challenge, Hugging Face evaluated Infinity, a containerized hardware-optimized inference solution paired with the Infinity Multiverse model optimization service. Testing covered 192 configurations on Amazon EC2 C6i instances powered by 3rd generation Intel Xeon Scalable processors across varying CPU cores, sequence lengths, and batch sizes. The benchmarks demonstrated that an Ice Lake-optimized DistilBERT container achieved up to 800% higher throughput than vanilla Transformers and delivered 1 to 4 millisecond end-to-end latencies for sequence lengths up to 64 tokens. Although Infinity was later discontinued in favor of Inference Endpoints and Optimum libraries, the results demonstrated substantial efficiency gains on modern CPU hardware.


### [Boosting Wav2Vec2 with n-grams in 🤗 Transformers](https://yomu.fyi/post/boosting-wav2vec2-with-n-grams-in-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Jan 12, 2022
- Type: Tutorial

Wav2Vec2 models fine-tuned with Connectionist Temporal Classification transcribe speech without external language models, but decoding can still suffer from spelling inaccuracies. Hugging Face Transformers addressed this by integrating Kensho Technologies' pyctcdecode library to support decoding with n-gram language models. Instead of decoding using simple argmax operations over logits, the Wav2Vec2ProcessorWithLM class feeds full probability matrices into beam search guided by KenLM n-gram probabilities. KenLM's build\_binary utility compresses language model ARPA files into binary formats, reducing file size by more than half for faster loading and hub deployment. In Swedish xls-r-300m-sv benchmarks on Common Voice 7, this 5-gram boosted decoding setup achieved an 18.85% word error rate, delivering an approximate 30% relative performance gain.


### [Nerd Out on 10 of Our Favorite Posts From 2021](https://yomu.fyi/post/nerd-out-on-10-of-our-favorite-posts-from-2021.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 29, 2021
- Type: Announcement

Shopify’s year-end roundup curates ten favorite engineering posts from 2021, spanning Ruby, storefronts, data analysis, databases, search, GraphQL, CI, APIs, and mobile development, while also noting several older reader favorites. The selection includes tutorials on building a web app with standard Ruby libraries, exploratory data analysis, GraphQL, Apache Beam for search, and React Native App Clips, alongside technical accounts of Hydrogen, YJIT, MySQL upgrades, and GraphQL rate limiting. YJIT was merged upstream and included in Ruby 3.1.0, while the CI team reduced the p95 of Shopify’s core monolith from 45 minutes to 18 minutes. The roundup also describes query-cost-based GraphQL rate limiting, rollback-testing roadblocks during a MySQL upgrade, and an App Clip effort that used a fail-fast, iterative approach.


### [Perceiver IO: a scalable, fully-attentional model that works on any modality](https://yomu.fyi/post/perceiver-io-a-scalable-fully-attentional-model-that-works-on-any-moda.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Niels Rogge
- Published: Dec 15, 2021
- Type: Announcement

Standard Transformer architectures scale poorly in compute and memory because pairwise dot-product self-attention depends quadratically on input size. Perceiver IO addresses this constraint by computing self-attention across a small set of latent variables rather than directly on high-dimensional inputs. Inputs and outputs interact with the model via cross-attention operations, decoupling compute and memory costs from input and output dimensions. Integrated into Hugging Face Transformers via the PerceiverModel class, the architecture supports diverse data types using optional preprocessors, decoders, and postprocessors. Experiments demonstrate competitive performance across text, multimodal video classification, 3D point cloud classification on ModelNet40, and StarCraft II reinforcement learning in AlphaStar.


[Newer posts](https://yomu.fyi/topic/performance/page/34.md) · [Older posts](https://yomu.fyi/topic/performance/page/36.md)