---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Announcing chDB 4: write Pandas, run ClickHouse, now on Hex](https://yomu.fyi/post/announcing-chdb-4-write-pandas-run-clickhouse-now-on-hex.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Ryadh Dahimene, Auxten Wang
- Published: Mar 11, 2026
- Type: Announcement

chDB 4 introduces Data Store, a native component that allows users to write Pandas-like code that executes on the ClickHouse OLAP SQL engine. Standard Pandas workflows suffer from eager execution, excessive memory materialization, and lack of automatic query optimizations. To solve this, Data Store implements a four-layer architecture that lazily records operations as a chain of LazyOp objects and compiles them into optimized ClickHouse SQL plans upon evaluation triggers. When encountering operations unsupported by ClickHouse, the QueryPlanner splits the pipeline into segments, routing tasks between ClickHouse and Pandas while using Python's memoryview to minimize data copying overhead. This release also integrates natively into Hex notebooks.


### [GitTrends: A Google Trends style view of the GitHub ecosystem](https://yomu.fyi/post/gittrends-a-google-trends-style-view-of-the-github-ecosystem.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Lionel Palacin
- Published: Mar 10, 2026
- Type: Announcement

GitHub produces billions of events across issues, pull requests, and comments, but turning this vast stream into a real-time trends analyzer presents significant technical challenges. GitTrends is an open-source demo application designed to search and compare technology keywords across nearly ten billion GitHub events. Built to showcase the full-text search capabilities recently released in ClickHouse, the application indexes raw text without pre-computed answers or intermediate data transformation. ClickHouse executes full-text searches and aggregations within a single engine in one pass, eliminating cross-system joins and data movement. The demo includes live comparisons toggling between full-text search, bloom filters, and full table scans to illustrate query performance at scale.


### [Announcing General Availability of ClickHouse Full-text Search](https://yomu.fyi/post/announcing-general-availability-of-clickhouse-full-text-search.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: Melvyn Peignon
- Published: Mar 10, 2026
- Type: Announcement

ClickHouse has reached general availability for its native Full-text Search feature, designed to accelerate token-based text filtering across large datasets. The capability relies on deterministic inverted indexes rather than probabilistic Bloom filter skip indexes, mapping individual tokens directly to row numbers. It supports plain strings, string arrays, and map keys or values while allowing configurable pre-processing and tokenization using standard SQL expressions. The system does not implement relevance scoring or store positional data for ranking, focusing instead on rapid filtering paired with large-scale analytical aggregations. Testing shows significant reductions in scanned granules alongside substantial latency improvements, though text indexes require more storage than Bloom filters.


### [Introducing Storage Buckets on the Hugging Face Hub](https://yomu.fyi/post/introducing-storage-buckets-on-the-hugging-face-hub.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Lucain Pouget, Eliott Coyac, Adrien Carreira, Victor Mustar, Julien Chaumond, Quentin Lhoest, Pierric Cistac, Sylvestre Bcht, Hugo Larcher, Rajat Arya, Di Xiao, Assaf Vayner
- Published: Mar 10, 2026
- Type: Announcement

Hugging Face introduces Storage Buckets as a mutable, S3-like object storage layer on the Hub for intermediate machine-learning artifacts such as checkpoints, optimizer states, processed shards, logs, and traces. Unlike versioned Models and Datasets repositories, Buckets support frequent writes, overwrites, directory synchronization, and cleanup while retaining standard Hub permissions and public or private visibility. Built on Xet, they chunk and deduplicate content across files, reducing repeated bytes, bandwidth, transfer time, and, for Enterprise customers, billed storage. The CLI and Python APIs support creation, syncing, dry runs, plans, inspection, individual file operations, and programmatic workflows; HfFileSystem and fsspec also allow compatible libraries to read and write through hf:// paths. Pre-warming can place hot data near AWS or GCP compute, while roadmap work aims to connect Buckets directly with versioned repos.


### [Keep the Tokens Flowing: Lessons from 16 Open-Source RL Libraries](https://yomu.fyi/post/keep-the-tokens-flowing-lessons-from-16-open-source-rl-libraries.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amine Dirhoussi, Quentin Gallouédec, Kashif Rasul, Lewis Tunstall, Edward Beeching, Albert Villanova del Moral, Nouamane Tazi, Leandro von Werra, Sergio Paniego
- Published: Mar 10, 2026
- Type: Benchmark

At-scale reinforcement-learning training is constrained by long, variable data-generation workloads that leave training GPUs idle, especially for reasoning and agentic models. The surveyed architecture separates inference and training onto distinct GPU pools, connects them through a rollout buffer, and synchronizes weights asynchronously so both stages can proceed concurrently. Sixteen open-source libraries are compared across orchestration, buffering, weight synchronization, staleness, partial rollouts, LoRA, and distributed backends; Ray appears in 8 of 16 orchestration implementations, while NCCL broadcast is the default weight-transfer method. The proposed TRL async trainer favors a lightweight bounded queue with per-token model\_version metadata, packed NCCL transfers, and experimental support for prefix-resume or abort-and-retry handling of rollouts interrupted by weight updates.


### [How Critical Manufacturing uses ClickHouse to bring real-time intelligence to the factory floor](https://yomu.fyi/post/how-critical-manufacturing-uses-clickhouse-to-bring-real-time-intellig.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Mar 10, 2026
- Type: Problem & solution

Critical Manufacturing faced performance and scaling bottlenecks while processing millions of factory floor events using a legacy Microsoft SQL Server analytics stack. Increasing data volumes caused historical queries to take hours, straining real-time dashboards and escalating operational maintenance. The engineering team replaced this architecture by migrating analytics workloads to ClickHouse connected directly to streaming Kafka topics. They implemented an ELT pattern alongside ReplacingMergeTree engines to deduplicate late-arriving sensor records and utilized TTL policies for automated lifecycle management. The resulting platform delivers sub-second analytical queries across historical and real-time operational data, expands into OpenTelemetry observability storage, and maintains embeddings for retrieval-augmented generation workloads.


### [Financial Benchmarks](https://yomu.fyi/post/financial-benchmarks.md)
- Company: [Ramp](https://yomu.fyi/company/ramp.md)
- Author: Kedar Thakkar, Anton Biryukov, Ashwin Kumar, Ryne Carbone
- Published: Mar 9, 2026
- Type: Benchmark

Ramp describes how it benchmarks large language models (LLMs) used in financial products against day-to-day tasks such as invoice extraction, financial-statement OCR, policy review, accounting autocoding, compliance detection, and fund routing. These benchmarks combine task-specific metrics with cost, latency, reasoning effort, human decisions, historical context, and ground-truth datasets where available. For contextual invoice OCR, perfect extraction requires every field to match the user's final bill, while financial-statement OCR uses a 1% relative-error threshold against transcriptions from over 500 real P&L documents. Results show application-specific trade-offs: Gemini 3 Flash is described as a cost-efficient leader for several visual and financial tasks, while Claude models lead some high-accuracy or low-miss-rate settings, and behavior can differ within one provider. The framework emphasizes Pareto trade-offs and continuous testing, with plans to expand from benchmarking current performance to hill climbing on capabilities.


### [What Replo learned optimizing 100+ billion events in ClickHouse](https://yomu.fyi/post/what-replo-learned-optimizing-100-billion-events-in-clickhouse.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Author: ClickHouse
- Published: Mar 9, 2026
- Type: Problem & solution

Replo built an analytics pipeline on ClickHouse to serve more than 4,000 Shopify merchants processing 3,000 to 5,000 events per second. The initial single-table architecture recalculated session metrics on the fly, leading to poor query efficiency as volume scaled. To speed up dashboard queries, the team implemented a mark-and-unmark pattern with SummingMergeTree and refreshable materialized views, but real-time deduplication and fractional attribution caused runaway query backlog loops. Replo resolved the issue by limiting real-time recomputations to a specialized table holding only the last 40 minutes of purchase-related events. This architecture allowed processing over 100 billion records while keeping query times fast and ingestion lag to approximately one minute.


### [Ulysses Sequence Parallelism: Training with Million-Token Contexts](https://yomu.fyi/post/ulysses-sequence-parallelism-training-with-million-token-contexts.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kashif Rasul, Stas Bekman
- Published: Mar 9, 2026
- Type: Explainer

Ulysses Sequence Parallelism addresses the memory and compute pressure of training language models on sequences extending beyond tens of thousands of tokens, where attention remains quadratic in sequence length even with FlashAttention. It shards tokens across GPUs, projects local queries, keys, and values, then uses two all-to-all operations to exchange data so each GPU computes a subset of attention heads before restoring sequence-sharded outputs. The post describes integrations with Accelerate, the Transformers Trainer, and TRL's SFTTrainer, including variable-length handling and weighted loss aggregation for uneven valid-token counts. In reported four-GPU tests, SP=4 reduced per-GPU memory sufficiently for 96K-token training on H100 80GB GPUs, reached 13,396 tokens per second at 64K, and matched non-SP canonical token-normalized loss within 0.000005 maximum absolute difference.


### [LeRobot v0.5.0: Scaling Every Dimension](https://yomu.fyi/post/lerobot-v0-5-0-scaling-every-dimension.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Steven Palma, Pepijn Kooijmans, Jade Choghari, Caroline Pascal, Khalil Meftah, Martino Russi, Nicolas Rabault, Michel Aractingi, Virgile BATTO, Thomas Wolf
- Published: Mar 9, 2026
- Type: Announcement

LeRobot v0.5.0 is a major release that expands supported hardware, policies, datasets, simulation, and its software foundation, following more than 200 merged PRs and more than 50 new contributors since v0.4.0. It adds full Unitree G1 humanoid support for locomotion, manipulation, teleoperation, and whole-body control, alongside OpenArm, Earth Rover, CAN-bus motor controllers, and other integrations. The policy additions include Pi0-FAST's autoregressive VLAs with FAST action tokenization, Real-Time Chunking for continuously blended predictions, Wall-X, X-VLA, SARM, and PEFT fine-tuning. For data and simulation workflows, streaming video encoding removes waits between recording episodes, while EnvHub loads environments from the Hugging Face Hub and IsaacLab-Arena supplies GPU-accelerated simulation. The codebase now requires Python 3.12, uses Transformers v5, supports installable third-party policy plugins, and adds remote Rerun visualization plus installation and documentation improvements.


### [Reclaiming Terabytes: Optimizing Android image caching with TLRU](https://yomu.fyi/post/reclaiming-terabytes-optimizing-android-image-caching-with-tlru.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Nguyen Van Minh
- Published: Mar 6, 2026
- Type: Problem & solution

Grab’s Android app cached runtime images with DiskLruCache, but its 100 MB size limit did not remove stale content while the cache remained below capacity, contributing substantially to device storage usage. The team developed Time-Aware Least Recently Used (TLRU), which combines LRU size eviction with time-based expiration using last\_accessed metadata and a TTL, while a minimum cache-size threshold protects a baseline of cached images. TLRU retains the 100 MB maximum and evicts entries when current\_time minus last\_accessed exceeds the TTL; the implementation also supports reading existing LRU journal files by ignoring timestamp suffixes, enabling safe rollback. Configuration was tuned through experiments using cache hit ratio, with a maximum acceptable decline of three percentage points. After production rollout, P95 total app size fell approximately 50 MB, while hit ratio stayed within target and infrastructure costs did not increase.


### [Bringing Robotics AI to Embedded Platforms: Dataset Recording, VLA Fine‑Tuning, and On‑Device Optimizations](https://yomu.fyi/post/bringing-robotics-ai-to-embedded-platforms-dataset-recording-vla-fine.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Gaetan Bahl, Enzo Ruedas, Tess Boivin
- Published: Mar 5, 2026
- Type: Tutorial

Deploying Vision–Language–Action policies on embedded robots is constrained by compute, memory, power, and real-time control requirements; synchronous inference can leave the arm idle and cause oscillatory corrections. NXP presents dataset-recording practices for a tea-bag placement task, including fixed cameras, controlled lighting, a gripper-mounted view, calibration backups, and diverse training and validation splits with recovery episodes. The example uses 120 episodes, three 640×480 cameras at 30 fps, batch size 8, and checkpoint selection based on training and validation success rather than loss alone. For deployment, the VLA is decomposed into vision, LLM backbone, and action-expert stages, with selective quantization and asynchronous scheduling constrained by inference time being shorter than action execution. On the i.MX 95, optimized ACT latency reached 0.32 seconds with 0.89 global accuracy, while ONNX FP32 ACT reached 0.96 global accuracy at 2.86 seconds.


### [Supabase Storage: major performance, security, and reliability updates](https://yomu.fyi/post/supabase-storage-major-performance-security-and-reliability-updates.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: fabrizio
- Published: Mar 5, 2026
- Type: Announcement

Supabase Storage has shipped updates live in all regions for projects using Storage, targeting performance at scale, security vulnerabilities, and reliability issues. As projects reached tens of millions of objects, concurrent uploads exposed write bottlenecks, certain SQL deletes could orphan files, and a crafted path could escape the file backend’s storage root. The object-listing rewrite removes the prefixes table, six triggers, and twelve helper functions, derives folder structure from the objects table with a hybrid skip-scan algorithm, and uses cursor-based pagination; deep pagination on 60 million rows is up to 14.8 times faster without a write penalty. Direct SQL deletes are blocked by default through a statement-level trigger, while a configurable 30-second timeout supports cancellation of disconnected clients via native Postgres protocol. Idempotent migrations, TUS lock-race prevention, orphan-scanner fixes, OpenTelemetry metrics, and adapter and API bug fixes are included, with no code changes required.


### [PRX Part 3 — Training a Text-to-Image Model in 24h!](https://yomu.fyi/post/prx-part-3-training-a-text-to-image-model-in-24h.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: David Bertoin, Roman Frigg, Jon Almazán
- Published: Mar 3, 2026
- Type: Benchmark

PRX Part 3 reports a 24-hour speedrun for training a text-to-image diffusion model, combining techniques that performed well in earlier experiments under a budget of about $1,500 on 32 H200 GPUs. The recipe uses x-prediction in pixel space with 32-pixel patches and a 256-dimensional initial bottleneck, followed by LPIPS and DINOv2 perceptual losses, TREAD token routing, REPA alignment with DINOv3, and Muon for matrix parameters while Adam handles the rest. Training ran for 100,000 steps at 512px with batch size 1,024, then 20,000 steps at 1024px with batch size 512 and no REPA, across three synthetic datasets. The resulting model is described as usable, with strong prompt following and consistent aesthetics, though texture glitches, anatomy issues, and difficulty with hard prompts remain. The post attributes these limitations mainly to undertraining and limited data diversity and presents the recipe as a foundation for further scaling.


### [2,000 robots walk into a shop: Simulated A/B testing (2026) - Shopify](https://yomu.fyi/post/2-000-robots-walk-into-a-shop-simulated-a-b-testing-2026-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 27, 2026
- Type: Benchmark

Shopify’s SimGym addresses the difficulty of evaluating storefront changes when merchants lack enough traffic for conventional A/B tests, by simulating shopping sessions with LLM-guided robots. Each robot runs in a cloud Chromium browser with a persona, budget, and intent, while a counterpart tests the alternate theme; the system uses real page representations rather than a mocked DOM. To support bursty, sequential workloads with 89K–127K accumulated tokens per session, Shopify self-hosted gpt-oss-120b on 48 NVIDIA B200 GPUs, combining MXFP4 expert quantization, FP8 KV-cache, FlashInfer kernels, async scheduling, and guided decoding. Production measurements report 80K to 88K tokens/second per B200 from async scheduling, a 57% reduction in HTTP/gRPC overhead, and MIG experiments yielding about 20% lower latency and 12% higher daily throughput without quality regression; speculative decoding is validated but queued.


### [Mixture of Experts (MoEs) in Transformers](https://yomu.fyi/post/mixture-of-experts-moes-in-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Aritra Roy Gosthipaty, Pedro Cuenca, merve, Ilyas Moutawwakil, Arthur Zucker, Sergio Paniego, Pablo Montalvo
- Published: Feb 26, 2026
- Type: Explainer

Mixture of Experts (MoEs) in Transformers let a Transformer retain its backbone while replacing selected feed-forward layers with routed expert sub-networks, allowing total model capacity to exceed the active parameters used per token. The post explains why sparse models can reduce inference cost and memory pressure, then focuses on making them first-class in Transformers. Its weight-loading refactor converts independently serialized expert tensors into packed runtime layouts through composable WeightConverter operations and lazy materialization. The Experts Backend supplies eager, batched\_mm, and grouped\_mm execution strategies, while expert parallelism shards experts across devices and aggregates partial outputs. Training support uses grouped GEMM and custom Triton plus LoRA kernels, with reported improvements of approximately 12× speed, over 35% VRAM reduction, and roughly 6× longer context.


### [The generative recommender behind Shopify's commerce engine](https://yomu.fyi/post/the-generative-recommender-behind-shopify-s-commerce-engine.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Feb 25, 2026
- Type: Explainer

Shopify describes a foundational generative recommender that predicts next products, and in some contexts ads, from buyer event sequences spanning searches, views, add-to-carts, favorites, and purchases. The system uses an autoregressive model with causal masking, time-aware attention combining RoPE-inspired rotary encoding with relative attention bias, and sampled softmax with shared and positive-aware hard negatives. Built on HSTU, it combines richer activity data, ensemble-focused training for incremental recall, and optimized CUDA kernels; the full training pipeline reached up to 7.3x the speed of a baseline implementation. In an online test of an August model version, relative shop orders rose 0.94%, high-quality click-through rate rose 5%, conversion rate rose 0.71%, and final served product recall at 2 rose 4.8%.


### [Custom Kernels for All from Codex and Claude](https://yomu.fyi/post/custom-kernels-for-all-from-codex-and-claude.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw, Sayak Paul, Aritra Roy Gosthipaty, shaun smith
- Published: Feb 13, 2026
- Type: Announcement

Writing CUDA kernels that integrate with diffusers and transformers involves GPU-specific optimization, library integration, PyTorch registration, and environment compatibility. It presents a cuda-kernels agent skill that packages architecture guides, integration patterns, kernel templates, troubleshooting material, benchmarks, and Kernel Hub examples for coding agents. Claude and Codex used it to generate RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for LTX-Video in diffusers, and an RMSNorm kernel for Qwen3-8B in transformers, including CUDA source, PyTorch bindings, build configuration, and benchmark scripts. On H100 80GB HBM3, the LTX-Video RMSNorm tests averaged 1.88x speedup and the optimized pipeline with torch.compile reached 1.43x the listed baseline, while Qwen3-8B RMSNorm averaged 1.94x and reached 2.47x at 8192 tokens. The skill supports packaging kernels for precompiled distribution through the Hub, where get\_kernel selects a matching binary across Python, PyTorch, and CUDA variants.


### [Transformers.js v4: Now Available on NPM!](https://yomu.fyi/post/transformers-js-v4-now-available-on-npm.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Joshua, Nico Martin
- Published: Feb 9, 2026
- Type: Announcement

Transformers.js v4 is now available on NPM, marking a major release after a year of development. It introduces a C++-rewritten WebGPU runtime tested across about 200 supported model architectures, enabling the same JavaScript code to run WebGPU-accelerated models in browsers, Node, Bun, Deno, and desktop applications. Specialized ONNX Runtime Contrib Operators support new export strategies for large language models, and the post reports a roughly fourfold speedup for BERT-based embedding models using MultiHeadAttention. The release also restructures the repository as a pnpm-workspaces monorepo, modularizes model definitions, moves examples to a separate repository, and switches from Webpack to esbuild, reducing build times from two seconds to 200 milliseconds and shrinking transformers.web.js by 53%. New APIs cover model asset inspection, caching, environment fetch controls, logging, and standalone tokenization, while added architectures include Mamba, MLA, and MoE models compatible with WebGPU.


### [Edge AI Made Easy: MongoDB and ObjectBox Data Synchronization](https://yomu.fyi/post/edge-ai-made-easy-mongodb-and-objectbox-data-synchronization.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Puja Roy
- Published: Feb 3, 2026
- Type: Announcement

Modern edge AI deployments require immediate real-time decision-making, local privacy, and offline operation while retaining the analytical scale of central cloud infrastructure. To address these hybrid demands, MongoDB and ObjectBox partnered to release the ObjectBox MongoDB Sync Connector as part of ObjectBox 5.0. ObjectBox operates as a lightweight on-device database with local vector search, multi-language bindings, and high-throughput I/O suited for resource-constrained hardware. In environments such as industrial factory floors and restaurant point-of-sale terminals, devices capture and evaluate operational data onsite without depending on active network connections. The native connector subsequently synchronizes critical operational datasets to MongoDB Atlas for long-term storage, fleet-wide monitoring, and machine learning model retraining.


[Newer posts](https://yomu.fyi/topic/performance/page/24.md) · [Older posts](https://yomu.fyi/topic/performance/page/26.md)