---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Building world-class product search at Shopify: Where C++ excellence meets ML innovation](https://yomu.fyi/post/building-world-class-product-search-at-shopify-where-c-excellence-meet.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 12, 2025
- Type: Problem & solution

Shopify describes a product-search stack built for multi-tenant commerce, where real-time indexing, retrieval, ranking, experimentation, and merchant controls must operate at global scale. Commerce ranking combines classical information retrieval with transformers, neural rankers, LightGBM, Catboost, and signals such as relevance, purchase popularity, brand trust, and navigational intent, while supporting privacy, multi-language, and accessibility requirements. RankFlow, a Python-like domain-specific language, lets data scientists change ranking logic without C++ expertise, while TurboDSL compiles and executes it with type safety, vectorized operations, cache optimization, predictable latency, and efficient memory use. The team first shipped SimScorerDSL in C++, then built TurboDSL without changing the DSL syntax, enabling continued experimentation during the transition. TurboDSL achieved a 48% speedup in ranking feature computation, and automated pull-request analysis compares component timings, statistical significance within a ±2% tolerance, and side-by-side results to detect performance or accuracy regressions.


### [Grab's Mac Cloud Exit supercharges macOS CI/CD](https://yomu.fyi/post/grab-s-mac-cloud-exit-supercharges-macos-ci-cd.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Madushan Gamage
- Published: Nov 6, 2025
- Type: Problem & solution

Grab moved its macOS CI/CD infrastructure from a US cloud vendor to a colocation cluster in Malaysia, closer to its Git servers in Southeast Asia. The change followed growth from one office Mac Pro to more than 250 Mac minis, rising demand, job queues, and cloud costs that made ownership or colocation more attractive at scale. Grab selected bare-metal hardware after comparing cloud, colocation, and on-premises options, then migrated progressively while monitoring failure rates, performance, and reliability; the later infrastructure also uses zero-touch provisioning to reduce manual setup as the cluster grows. The migration produced projected savings of more than $2.4 million over three years and performance improvements of 20–40%, while the team chose allocated-kilowatt power commitments and retained bare metal because macOS virtualization’s performance or stability trade-offs did not justify it.


### [Upgrade to Puma 7 and Unlock the Power of Fair Scheduled Keep-alive](https://yomu.fyi/post/upgrade-to-puma-7-and-unlock-the-power-of-fair-scheduled-keep-alive.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Richard Schneeman
- Published: Nov 5, 2025
- Type: Explainer

Puma 6 and earlier could let a keep-alive connection “cut the line” when all threads were busy, allowing up to ten requests from that connection to run consecutively while previously accepted work waited. The behavior came from reusing a “fast inline” optimization intended for HTTP pipelining, where requests arrive together, for keep-alive requests that can arrive later. Puma 7 changes queueing so pipelined and keep-alive requests return to the back of the queue, and adjusts wait\_until\_not\_full so interleaving is possible; its accept loop now slows proportionally with load instead of stopping completely. The fix does not add capacity, but distributes overload delay more fairly across requests. The post recommends upgrading to Puma 7.1.0 or later, especially for applications that had disabled keep-alives after latency spikes.


### [Improving performance by prefetching product pages from Etsy Search](https://yomu.fyi/post/improving-performance-by-prefetching-product-pages-from-etsy-search.md)
- Company: [Etsy](https://yomu.fyi/company/etsy.md)
- Author: David Weinzimmer
- Published: Oct 29, 2025
- Type: Problem & solution

Etsy’s search and web performance teams tested the Speculation Rules API (SRA) to reduce loading time for product listing pages reached from desktop organic search. They added a speculationrules configuration that prefetches eligible listing HTML after a 200-millisecond hover, using URL and data-attribute selectors with moderate eagerness; unlike link prefetch, SRA uses both memory and HTTP caches. The experiment required attention to a two-page memory limit, five-minute retention, non-cacheable listing HTML, video hover behavior, cookies, new-tab targets, and analytics events triggered during prefetches. Results showed 23.6% faster 75th-percentile TTFB, with improvements of 20.7% in FCP, 21.1% in LCP, 20.4% in DOMContentLoaded, and 10.6% in Page Load, while roughly one in 14 prefetches led to navigation.


### [Composer: Building a fast frontier model with RL](https://yomu.fyi/post/composer-building-a-fast-frontier-model-with-rl.md)
- Company: [Cursor](https://yomu.fyi/company/cursor.md)
- Author: Cursor
- Published: Oct 29, 2025
- Type: Announcement

Composer is a mixture-of-experts agent model for software engineering that targets interactive use, with reported frontier coding results and generation speed four times faster than similar models on the authors’ benchmarks. It is trained with reinforcement learning on real-world software engineering challenges in large codebases, using production search and editing tools, terminal commands, and codebase-wide semantic search. Training responses may be code edits, plans, or informative answers, while the process rewards efficient tool use, parallelism, minimal unnecessary responses, and evidence-based claims. Evaluation uses Cursor Bench, built from real agent requests and curated optimal solutions, measuring correctness alongside adherence to existing abstractions and software engineering practices. The training stack combines asynchronous reinforcement learning with PyTorch and Ray, MXFP8 MoE kernels, expert parallelism, hybrid sharded data parallelism, and sandboxed cloud environments scaled to hundreds of thousands concurrently.


### [Streaming datasets: 100x More Efficient](https://yomu.fyi/post/streaming-datasets-100x-more-efficient.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Andres Marafioti, Quentin Lhoest, ben burtenshaw, Pedro Cuenca, merve
- Published: Oct 27, 2025
- Type: Problem & solution

Training machine learning models on multi-terabyte datasets often requires long download times or suffers from network request storms when multiple DataLoader workers initialize independently. Hugging Face revamped the backend of its datasets library to make dataset streaming viable for large-scale distributed training without modifying the standard API. Startup latency was reduced by caching resolved data file lists across workers and bundling API calls, cutting initial request volume up to one hundred times. The team also implemented background prefetching for Parquet files and added configurable buffering parameters to keep GPUs fully saturated during training runs. Benchmark tests demonstrated up to twice the sample processing throughput and zero worker crashes across 256 concurrent workers on 64 H100 GPUs.


### [LeRobot v0.4.0: Supercharging OSS Robot Learning](https://yomu.fyi/post/lerobot-v0-4-0-supercharging-oss-robot-learning.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Steven Palma, Michel Aractingi, Pepijn Kooijmans, Caroline Pascal, Jade Choghari, Francesco Capuano, Adil Zouitine, Martino Russi, Thomas Wolf
- Published: Oct 24, 2025
- Type: Announcement

LeRobot v0.4.0 expands the open-source robot-learning stack with scalable datasets, simulation benchmarks, processing tools, policies, and hardware integrations. LeRobotDataset v3.0 introduces chunked episodes, video streaming, unified Parquet metadata, improved loading and memory efficiency, plus utilities for editing and converting datasets. A modular Processor system separates policy and robot pipelines, while direct Accelerate integration simplifies multi-GPU training; the release also adds LIBERO, Meta-World, PI0, PI0.5, and GR00T N1.5 support. A plugin architecture enables third-party robots, cameras, and teleoperators to be installed without changing the core library, with Reachy 2 and phone teleoperation included, alongside a self-paced Robot Learning Course and hands-on tutorial.


### [Google Cloud C4 Brings a 70% TCO improvement on GPT OSS with Intel and Hugging Face](https://yomu.fyi/post/google-cloud-c4-brings-a-70-tco-improvement-on-gpt-oss-with-intel-and.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Jiqing.Feng, Matrix Yao, Ke Ding, Ilyas Moutawwakil
- Published: Oct 16, 2025
- Type: Benchmark

Intel and Hugging Face benchmarked OpenAI GPT OSS text generation to evaluate performance on Google Cloud C4 virtual machines powered by Intel Xeon 6 processors against previous-generation C3 instances. The benchmark evaluated the unsloth/gpt-oss-120b-BF16 Mixture of Experts model across batch sizes from 1 to 64 using bfloat16 precision, static KV cache, and SDPA attention. To eliminate redundant computation, Hugging Face merged an optimization directing each expert sub-network to process only its routed tokens rather than all tokens. Testing demonstrated that C4 instances deliver 1.4x to 1.7x higher throughput per vCPU compared to C3 instances. At a batch size of 64, C4 achieved a 1.7x Total Cost of Ownership improvement alongside lower hourly pricing.


### [Get your VLM running in 3 simple steps on Intel CPUs](https://yomu.fyi/post/get-your-vlm-running-in-3-simple-steps-on-intel-cpus.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Ezequiel Lanza, Helena, Nikita, Ella Charlaix, Ilyas Moutawwakil
- Published: Oct 15, 2025
- Type: Tutorial

Deploying Vision Language Models locally presents computational challenges despite providing data privacy and independence from external servers. Optimum Intel and OpenVINO address this by optimizing lightweight models like SmolVLM2-256M for low-resource hardware, including Intel CPUs and GPUs. The process involves exporting the model to the OpenVINO Intermediate Representation format, applying 8-bit weight-only or static quantization, and executing inference with the transformed model. In single-image CPU benchmarks on an Intel Core Ultra 7 265K processor, OpenVINO with 8-bit weight-only quantization reduced time to first token from 5.150 seconds under PyTorch down to 0.247 seconds. The optimized configuration also raised decoding throughput from 0.722 to 63.928 tokens per second.


### [The Cost of Not Knowing MongoDB, Part 3: appV6R0 to appV6R4](https://yomu.fyi/post/the-cost-of-not-knowing-mongodb-part-3-appv6r0-to-appv6r4.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Artur Costa
- Published: Oct 9, 2025
- Type: Benchmark

To alleviate a disk throughput bottleneck on a MongoDB server, this optimization pass focuses on shrinking document sizes across application revisions appV6R0 through appV6R4. The implementation replaces an array of items with a dynamic schema document, using date segments such as days or combined months and days as field names to store status totals. Redundant date components already encoded in the binary \_id field are omitted from internal document keys. Bulk upsert operations rely on updateOne with the $inc operator to update matching date keys or create documents when missing. Aggregation pipelines process reports using the $objectToArray and $reduce operations to calculate totals across date intervals while preserving a single index on \_id.


### [Highly concurrent in-memory counter in GoLang](https://yomu.fyi/post/highly-concurrent-in-memory-counter-in-golang.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Naveen Kumar Jakuva Premkumar
- Published: Oct 6, 2025
- Type: Problem & solution

An overloaded database faced heavy traffic from campaign usage tracking, motivating an in-memory counter rather than an immediate SQL-to-NoSQL migration. Because exact counts were not required, increments could be cached in the application server and flushed periodically to reduce database queries and CPU use. The implementation uses Go’s Sync.Map, with LoadOrStore and CompareAndSwap for concurrent increments, plus LoadAndDelete to atomically remove values for asynchronous persistence; the source notes a fresh-map swap as an alternative locking optimization. On an Apple M4 Pro, the 2,000-key benchmark reported 53 ns/op and 1 billion operations per 60 seconds, versus 160 ns/op and 466 million operations for a Mutex-protected map, but Sync.Map used 88 B/op and four allocations. In production, update queries fell from 140 QPS to 45 QPS, master CPU from 35% to 18%, and replica CPU from 19% to 12%.


### [The 10 Skills I Was Missing as a MongoDB User](https://yomu.fyi/post/the-10-skills-i-was-missing-as-a-mongodb-user.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Joel Lord
- Published: Oct 2, 2025
- Type: Explainer

Transitioning from relational databases to MongoDB often leads developers to replicate SQL habits, causing severe performance bottlenecks, brittle schemas, and operational challenges. Effective document design requires balancing referencing and embedding based on query access patterns and update frequencies, rather than placing every entity into separate collections or excessively nesting large documents. Database performance depends on matching indexes to specific query shapes and field orders, which developers can systematically verify by inspecting execution plans using explain plans. Utilizing the built-in aggregation framework allows filtering, grouping, and calculations to run natively inside the database, eliminating slow, manual data transformations in application code. Establishing early monitoring for latency, replication lag, memory usage, and cluster failover resilience ensures stability before adopting advanced tools like Atlas Search and Vector Search.


### [SOTA OCR with Core ML and dots.ocr](https://yomu.fyi/post/sota-ocr-with-core-ml-and-dots-ocr.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Christopher Fleetwood, Pedro Cuenca
- Published: Oct 2, 2025
- Type: Tutorial

Running competitive models like RedNote's 3B parameter dots.ocr on-device provides zero-cost, network-free optical character recognition while leveraging power-efficient hardware like Apple's Neural Engine. Because the Neural Engine requires Core ML, converting the 1.2B parameter NaViT vision encoder from PyTorch requires resolving multiple graph tracing incompatibilities. The initial conversion pipeline targets FLOAT32 execution on GPU by capturing the execution graph with torch.jit.trace and compiling it via coremltools. Simplifying the model for single-image inference eliminates dynamic sequence masking, dynamic tensor iteration, and multi-attention complexity. Although the resulting Core ML model accurately matches PyTorch baseline numerical precision, the unoptimized artifact spans over 5GB and takes over one second per forward pass.


### [Accelerating Qwen3-8B Agent on Intel® Core™ Ultra with Depth-Pruned Draft Models](https://yomu.fyi/post/accelerating-qwen3-8b-agent-on-intel-coretm-ultra-with-depth-pruned-dr.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Igor Margulis, Ofir Zafrir, Shira Guskin, Guy Boudoukh, Pedro Cuenca
- Published: Sep 29, 2025
- Type: Problem & solution

Local agentic AI workflows generate multi-step reasoning traces that increase token counts and strain real-time inference on personal computers. To accelerate execution, researchers implemented speculative decoding using OpenVINO.GenAI on an Intel Lunar Lake integrated GPU, pairing a 4-bit Qwen3-8B target model with a smaller Qwen3-0.6B draft model. By measuring angular distance across layers, the team pruned six of the draft model's twenty-eight layers to minimize draft latency and fine-tuned it on synthetic data generated from 500,000 prompts. This depth-pruned draft elevated speculative decoding throughput from a 1.3x speedup to a 1.4x speedup over the standalone baseline. Finally, deploying the optimized model alongside the Hugging Face smolagents framework allowed local agents to perform tool invocation and code generation tasks efficiently.


### [MongoDB SQL Interface: Now Available for Enterprise Advanced](https://yomu.fyi/post/mongodb-sql-interface-now-available-for-enterprise-advanced.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Jourdain Patrick, Alexi Antonino
- Published: Sep 25, 2025
- Type: Announcement

MongoDB has released the general availability of the MongoDB SQL Interface for MongoDB Enterprise Advanced on self-managed deployments running MongoDB 6.0 or higher. Previously, teams relying on SQL-based business intelligence tools faced maintenance bottlenecks when learning MongoDB Query Language or creating custom extract, transform, and load pipelines. The new interface translates standard SQL queries into native MongoDB operations in real time using MongoSQL, a SQL-92 compatible dialect. Users utilize a Schema Builder CLI to generate JSON schemas that map nested document structures and store them alongside data for query validation. This native query translation replaces the MongoDB BI Connector, which will remain supported until September 2026, while offering improved performance and direct connectivity via JDBC and ODBC drivers.


### [MongoDB Community Edition to Atlas: A Migration Masterclass With BharatPE](https://yomu.fyi/post/mongodb-community-edition-to-atlas-a-migration-masterclass-with-bharat.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Nick Bell
- Published: Sep 21, 2025
- Type: Problem & solution

BharatPE, an Indian fintech processing over ₹12,000 crore monthly across 450 cities, operated a 45-terabyte self-hosted MongoDB Community Edition deployment. Managing three sharded three-node clusters resulted in uneven data distribution, high maintenance costs, scaling friction, and disaster recovery gaps under strict regulatory requirements. To modernize infrastructure, the database operations team partnered with MongoDB to execute a structured five-stage migration to MongoDB Atlas. The initiative utilized mongosync for secure data transition alongside environment mirroring and automated integrity validation scripts. Transitioning to MongoDB Atlas yielded a 40 percent improvement in query response times and provided automated failover backed by a 99.995 percent uptime SLA.


### [Modernizing Core Insurance Systems: Breaking the Batch Bottleneck](https://yomu.fyi/post/modernizing-core-insurance-systems-breaking-the-batch-bottleneck.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Vinod Bagal, Jagpreet Singh
- Published: Sep 18, 2025
- Type: Problem & solution

Migrating core insurance platforms from legacy relational databases with PL/SQL to Java and MongoDB Atlas frequently degrades batch processing performance. Like-for-like migrations can cause batch jobs to run 25 to 30 times slower due to high network round-trips, inefficient per-record operations, and under-utilized bulk capabilities. To resolve these bottlenecks, an optimization framework was developed that leverages MongoDB native bulkWrite operations, intelligent reference data prefetching, and parallel execution. An architecture combining a Spring Boot controller and a dedicated executor framework partitions workloads across threads or event processors. This framework recovered lost performance, completing previously failing batch jobs within service-level agreements and achieving 10 to 15 times better execution speeds than legacy systems in certain workloads.


### [MongoDB.local NYC 2025: Defining the Ideal Database for the AI Era](https://yomu.fyi/post/mongodb-local-nyc-2025-defining-the-ideal-database-for-the-ai-era.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Dev Ittycheria, President and CEO, MongoDB
- Published: Sep 18, 2025
- Type: Announcement

At MongoDB.local NYC, MongoDB introduced new database capabilities, AI integrations, and modernization tools designed for agentic and generative AI workloads. The release of MongoDB 8.2 arrives alongside public previews of Search and Vector Search for both MongoDB Community Edition and Enterprise Server. MongoDB also highlighted Voyage AI embedding models and rerankers to improve retrieval accuracy across raw data, metadata, and embeddings. To address the costs and constraints of legacy infrastructure, the new MongoDB Application Modernization Platform combines AI-assisted tooling and specialized expertise to migrate legacy systems. According to MongoDB, early migrations using this platform run two to three times faster, while code rewriting tasks accelerate by an order of magnitude.


### [Taming the monorepo beast: Our journey to a leaner, faster GitLab repo](https://yomu.fyi/post/taming-the-monorepo-beast-our-journey-to-a-leaner-faster-gitlab-repo.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Nagendra Gangwar
- Published: Sep 16, 2025
- Type: Problem & solution

Grab’s decade-old Go monorepo grew to 12.7 million commits, 73GB of Git-tree metadata, 176GB of blobs, 12 million references, and 444,000 files, causing replication lag, slow developer workflows, and GitLab operational bottlenecks. A staging proof of concept compared full and single-commit histories under production-like load: replication delay fell from 160–240 seconds to 1–2.5 seconds, nearly a 100x improvement. Because custom migration tooling processed about 100 commits per hour, the team retained commits reachable from more than 2,000 critical tags, one month of recent history, and simplified merge commits, while archiving older history. The resulting prune reduced commits by 99.9%, improved replication performance by 99%, preserved code and tags without data loss, and left teams with minor migration adjustments.


### [How we built it: Real-time analytics for Stripe Billing](https://yomu.fyi/post/how-we-built-it-real-time-analytics-for-stripe-billing.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Reed Trevelyan
- Published: Sep 16, 2025
- Type: Problem & solution

Stripe built a real-time streaming analytics system for Billing after customers said they needed to detect emerging behavior and adapt pricing quickly, replacing a batch pipeline whose subscription updates lagged by 24 hours on average. The architecture uses Apache Flink to maintain compressed subscription history and incrementally process analytics events, while a custom Apache Spark job generates initial Flink state from billions of historical events and supports validation and exports. A new Apache Pinot v2 query engine performs windowed aggregations and joins without offline preaggregation; most updates are processed in under one minute, nearly all reach users within 15 minutes, and production query latency is under 300 milliseconds. Metric-definition changes trigger historical recalculation while new events are buffered and later replayed, preserving a consistent Dashboard view during the transition.


[Newer posts](https://yomu.fyi/topic/performance/page/26.md) · [Older posts](https://yomu.fyi/topic/performance/page/28.md)