---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Scaling AI Inference Across Multiple GPUs Using NVIDIA TensorRT with Multi-Device Inference Support](https://yomu.fyi/post/scaling-ai-inference-across-multiple-gpus-using-nvidia-tensorrt-with-m.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Peter Kisfaludi
- Published: Jun 25, 2026
- Type: Tutorial

Generative AI media pipelines can exceed the memory and compute capacity of a single GPU, creating a deployment challenge for models that retain TensorRT’s kernel fusions, memory planning, and quantization. TensorRT 11.0 introduces native multi-device inference and IDistCollectiveLayer primitives, while NCCL supplies topology-aware transport across NVLink, NVSwitch, PCIe, and InfiniBand for supported distributed collectives. The article contrasts tensor parallelism, which shards layer weights, with context parallelism, which partitions sequences; it then focuses on AllGather KV, Ring Attention, and DeepSpeed Ulysses for long-sequence media attention. It also describes converting PyTorch pipelines with Torch-TensorRT into C++-deployable engines, benchmarking Cosmos 3 and FLUX.1 on an eight-GPU node, and provides C++ and OpenMPI steps for configuring an eight-rank NCCL deployment.


### [Q&A: How KRAFTON Built PUBG Ally, a Co-Playable Character Powered by NVIDIA ACE](https://yomu.fyi/post/q-a-how-krafton-built-pubg-ally-a-co-playable-character-powered-by-nvi.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jun 25, 2026
- Type: Explainer

KRAFTON developed PUBG Ally, an on-device AI co-playable character for PUBG: BATTLEGROUNDS, using NVIDIA ACE tooling and models. Conventional non-playable characters rely on fixed dialogue, but PUBG Ally processes live game states and player voice input in real time. The interactive pipeline combines automatic speech recognition, a quantized Mistral-NeMo-Minitron-2B small language model, and a custom text-to-speech model running locally on client GPUs with at least 8GB of VRAM. To minimize latency, the architecture separates reactive gameplay into a behavior-tree System 1 layer while routing strategic decisions and speech generation to the language model. KRAFTON tuned the non-deterministic system across multi-layer evaluations and large-scale playtests with over a thousand players.


### [Achieving Near-Linear Training Scalability for Pinterest’s Foundation Models](https://yomu.fyi/post/achieving-near-linear-training-scalability-for-pinterest-s-foundation.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Jun 25, 2026
- Type: Problem & solution

Pinterest’s foundation ranking models store approximately 99% of their parameters in embedding tables, causing cross-node NCCL all-to-all communication to severely bottleneck multi-node distributed training. Initial multi-node scaling stalled at 1.21x on four nodes even after adopting AWS Elastic Fabric Adapter for OS-bypass networking. To resolve this, engineers profiled GPU activity with PyTorch Profiler and systematically addressed communication bottlenecks across multiple layers. The team implemented FP8 quantized communications using FBGEMM, balanced table sharding across GPUs, halved embedding dimensions while doubling row counts, and adopted a 2D parallel communication topology alongside PyTorch Distributed Checkpoint. These combined interventions achieved near-linear scaling factors of 3.9x on four nodes and 7.5x on eight nodes across 64 GPUs.


### [Why we rewrote WAL-G for Postgres backups in Rust: Meet WAL-RUS](https://yomu.fyi/post/why-we-rewrote-wal-g-for-postgres-backups-in-rust-meet-wal-rus.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 25, 2026
- Type: Problem & solution

ClickHouse Cloud built WAL-RUS, an open-source Rust implementation of PostgreSQL backup and WAL archival tooling, after encountering difficulty predicting WAL-G’s memory usage in resource-constrained environments. The project preserves WAL-G’s core functionality and compatibility while using bounded worker pools, controlled concurrency, persistent object-storage connections, and a streaming pipeline that limits buffering and data copies. It uses the same WALG\_ configuration variables, and WAL-G and WAL-RUS can read each other’s archives. In a sustained WAL-heavy benchmark with four archival workers, WAL-RUS stayed below 1 GB of peak virtual memory versus nearly 2.8 GB for WAL-G, while both maintained minimal backlog and comparable CPU usage. The authors plan to make WAL-RUS the default backup and archival mechanism for ClickHouse Cloud’s managed Postgres offering and are working to upstream its Postgres 17 wal summaries support to WAL-G.


### [How Vibe.co handles billions of ad impressions with ClickHouse Cloud](https://yomu.fyi/post/how-vibe-co-handles-billions-of-ad-impressions-with-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 25, 2026
- Type: Problem & solution

Vibe.co, recently acquired by Walmart, uses ClickHouse Cloud to provide real-time campaign reporting for thousands of Connected TV advertisers across billions of ad impressions. It migrated from Postgres after aggregation-heavy OLAP queries made its pre-aggregation layer a bottleneck, limiting interactive slicing and requiring new jobs for reporting use cases. The team evaluated self-hosted ClickHouse, another managed option, and ClickHouse Cloud, choosing the latter for operational simplicity, support from ClickHouse engineers, and separation of storage and compute with more predictable costs. Vibe uses refreshable materialized views for hot/cold tiering, keeping the last 30 days in a hot table; stored data grew from about 100 GB to over 2 TB without rearchitecting, while more than 90% of client campaign reports respond in under 100ms.


### [Accelerating BEV Pooling on NVIDIA GPUs for Physical AI Applications](https://yomu.fyi/post/accelerating-bev-pooling-on-nvidia-gpus-for-physical-ai-applications.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: John Yang
- Published: Jun 24, 2026
- Type: Problem & solution

Bird's-eye-view (BEV) pooling frequently becomes an inference latency bottleneck in spatial AI pipelines due to irregular memory access patterns and scatter-reduce operations. BEVPoolV3 addresses this by tailoring kernel execution to the GPU memory regime, specifically distinguishing between DRAM-bound and L2-cache-resident working sets. The implementation reduces redundant scatter traffic via depth-outer traversal, precomputes indices to eliminate runtime integer division, minimizes duplicate depth loads, and adopts interval-owned writes to bypass atomic operations. On a canonical 49 MB nuScenes working set, BEVPoolV3 accelerates TensorRT plugin execution from 274.0 µs down to 16.4 µs on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Furthermore, profiling with Nsight Compute reveals that L2-resident scatter-reduce operators achieve optimal efficiency with FP8, avoiding the ALU instruction overhead of NVFP4 decoding.


### [How Visa went from multi-day reporting to conversational analytics agents with ClickHouse Cloud and LibreChat](https://yomu.fyi/post/how-visa-went-from-multi-day-reporting-to-conversational-analytics-age.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 24, 2026
- Type: Problem & solution

Visa used ClickHouse Cloud to modernize reporting on Authorize.net payments data, replacing multi-day analyst-led turnaround with natural-language queries for internal teams. The solution pairs LibreChat, an open-source conversational interface, with the ClickHouse MCP server and runs on ClickHouse Cloud on AWS, allowing agents to explore and query a roughly 40 PB data footprint containing millions of rows. ClickHouse materialized views pre-compute and incrementally update results, reducing scanned data and bringing example analyses down to 100–200 milliseconds, with reported speedups of roughly 30–70 times. The platform also uses encrypted transfers, Amazon S3, VPC PrivateLink, change data capture, and federated access controls, while Visa reports identifying millions of dollars in revenue risk and reclaiming 8–10 hours per user each week.


### [Announcing the Managed ClickStack MCP Server](https://yomu.fyi/post/announcing-the-managed-clickstack-mcp-server.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 24, 2026
- Type: Announcement

Managed ClickStack in ClickHouse Cloud now includes the ClickStack MCP server, extending its observability investigation capabilities to external agents and AI workflows. Unlike the generic ClickHouse MCP server’s SQL-oriented interface, it offers semantic tools for log patterns, outlier correlation, slow traces, and repeatable investigations across logs, metrics, and traces. For event patterns, the server samples matching events, processes them with Drain3 in the MCP layer, and extrapolates and ranks counts, reducing output cardinality and making query overhead fixed rather than linear in the full dataset. Internal benchmarks report 25% fewer tool calls, 2.5x greater consistency, and an almost 20% increase in evaluation scores versus the standard ClickHouse MCP. Managed deployments also support full-fidelity telemetry, dedicated compute for agent workloads, and orchestration such as persisted dashboards, while native SQL remains available for unusual or deeper questions.


### [Maximize AI Factory Energy Efficiency Through Full-Stack Inference and Training Optimizations](https://yomu.fyi/post/maximize-ai-factory-energy-efficiency-through-full-stack-inference-and.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Sachin Idgunji
- Published: Jun 23, 2026
- Type: Explainer

Power costs comprise up to 40% of AI factory operating expenses, making performance per watt critical within fixed site power limits. Operators maximize throughput by combining architectural enhancements, narrow-precision formats like NVFP4, and mixture-of-experts model designs that activate fewer parameters per token. During distributed LLM training in Megatron-LM, tuning individual GPU speeds along critical execution paths eliminates idle power waste without extending total training time. Additionally, the NVIDIA DSX platform orchestrates dynamic power allocation, 45°C direct-to-chip liquid cooling, and grid integration across racks and facilities. Together, these full-stack optimizations enable performance-optimized AI factories to deliver up to 2.6 times more tokens per second per megawatt.


### [Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding](https://yomu.fyi/post/boost-inference-performance-up-to-15x-on-nvidia-blackwell-using-dflash.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Amr Elmeleegy
- Published: Jun 23, 2026
- Type: Benchmark

Sequential token generation in autoregressive LLMs limits GPU utilization and serving throughput, while conventional speculative decoding remains constrained by autoregressive draft models. DFlash addresses this bottleneck by employing a lightweight block-diffusion drafter that predicts an entire block of masked candidate tokens in a single forward pass. The architecture incorporates target hidden-state conditioning and key-value injection into draft model projections while relying on the target model for parallel verification. Evaluated on NVIDIA Blackwell GPUs across TensorRT-LLM, vLLM, and SGLang, DFlash increases gpt-oss-120b throughput by more than 15x at high interactivity targets and delivers up to 5.8x speedups on Gemma 4 31B compared to autoregressive decoding.


### [What's New in pg\_clickhouse v0.3.2: Postgres 19, TLS, Regex, and Memory](https://yomu.fyi/post/what-s-new-in-pg-clickhouse-v0-3-2-postgres-19-tls-regex-and-memory.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 23, 2026
- Type: Announcement

pg\_clickhouse v0.3.2 updates the Postgres interface for querying ClickHouse, with changes centered on PostgreSQL 19 Beta1 support, TLS configuration, regular-expression pushdown, and memory consumption. The release adds CREATE SERVER options for forcing TLS or plaintext, selecting a minimum TLS version, and enabling ClickHouse native-protocol compression. Revised pushdown logic maps Postgres regular-expression flags to ClickHouse behavior, while regexp\_match() can now be pushed down when its pattern has no capturing groups. Two memory issues were fixed: one involving unbuffered HTTP-driver queries and another involving repeated foreign scans such as parameterized inner scans in nested-loop joins. The minor update requires no reload, restart, or ALTER EXTENSION UPDATE, and also fixes empty-array ANY() errors on ClickHouse versions before 25.


### [ParallelKernelBench: Frontier LLMs can't write fast multi-GPU kernels (yet)](https://yomu.fyi/post/parallelkernelbench-frontier-llms-can-t-write-fast-multi-gpu-kernels-y.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Jun 23, 2026
- Type: Benchmark

ParallelKernelBench (PKB) measures whether language models can replace PyTorch + NCCL implementations with fast CUDA kernels for multi-GPU workloads, where communication can account for over 20% of inference latency. Its 87 problems come from real codebases and span tensor, context, data, expert, sequence, and FSDP/ZeRO parallelism, with hardware topology supplied and symmetric memory used for direct GPU communication over NVLink. In zero-shot evaluation, GPT-5.5 solved 28 problems and produced 22 faster-than-baseline solutions; sampling three attempts raised those figures to 36 and 27, but the best fast 1@3 result was only 31%. Models performed best on familiar collective primitives, tensor-parallel GEMMs, and Ulysses-style context parallelism, while failures often involved rank coordination, data partitioning, collective ordering, and underuse of TMA or NVLS.


### [Experimenting with the proposed Cross-Origin Storage API in Transformers.js](https://yomu.fyi/post/experimenting-with-the-proposed-cross-origin-storage-api-in-transforme.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Thomas Steiner
- Published: Jun 23, 2026
- Type: Tutorial

Transformers.js browser apps currently redownload and recache identical AI model and WebAssembly resources when they are requested from different origins. Chrome's cache uses a Network Isolation Key composed of the top-level site and current-frame site, so matching CDN URLs do not produce cache hits; the toy example incurs 177 MB of duplicate model storage, while a shared ONNX Runtime Wasm file is 4,733 kB. The proposed Cross-Origin Storage (COS) API addresses this by identifying large files with cryptographic hashes and sharing them across opted-in origins, with a polyfill extension enabling experimentation before native browser support. In the extended Transformers.js flow, a COS lookup can serve resources in milliseconds, while a miss downloads and stores them, and ModelRegistry.is\_pipeline\_cached() can help select among models already cached.


### [The end-to-end cost-performance of real-time analytics: Snowflake vs. ClickHouse Cloud](https://yomu.fyi/post/the-end-to-end-cost-performance-of-real-time-analytics-snowflake-vs-cl.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 23, 2026
- Type: Benchmark

CostBench compares Snowflake and ClickHouse Cloud as end-to-end real-time analytics systems rather than measuring reads on a static, prepared dataset. It evaluates continuous ingest, raw-data organization, pre-aggregation freshness, and continuous query serving while background work continues, testing Snowflake’s standard-table and Interactive Tables paths alongside ClickHouse. The benchmark is designed around fresh data becoming query-ready as it arrives and distinguishes sustained real-time ingest from bulk loading or backfill. In the reported results, ClickHouse achieved 969× better cost-performance than Snowflake’s standard materialized-view setup and 180× better than its Interactive Tables setup; among the tested paths, only ClickHouse incremental materialized views kept pre-aggregations continuously synchronized without lag.


### [CCCL Runtime: A Modern C++ Runtime for CUDA](https://yomu.fyi/post/cccl-runtime-a-modern-c-runtime-for-cuda.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Piotr Ciolkosz
- Published: Jun 22, 2026
- Type: Announcement

NVIDIA CCCL runtime introduces modern C++ abstractions for fundamental CUDA programming concepts starting in CUDA Toolkit 13.2 and CCCL 3.2. Designed as an alternative to the traditional CUDA runtime API, the library eliminates implicit global state by requiring explicit dependencies, such as binding streams directly to specific device references. The API enforces strong typing through owning types and non-owning reference types, omits default stream usage to ensure all streams remain non-blocking, and handles errors via standard C++ exceptions. In addition, CCCL runtime simplifies kernel launches through kernel functors that enable automatic template argument deduction and automatically transforms owning buffer arguments into device spans.


### [Enable Real-Time AI for High-Speed Data Acquisition with DAQIRI](https://yomu.fyi/post/enable-real-time-ai-for-high-speed-data-acquisition-with-daqiri.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Cara Laasch
- Published: Jun 22, 2026
- Type: Announcement

Scientific instruments and high-bandwidth sensors generate immense data volumes, but traditional architectures rely on a store-then-analyze model that bottlenecks real-time discovery and forces extensive data discarding. NVIDIA DAQIRI shifts data acquisition to a software-centric pipeline that connects streaming detectors directly to GPU computing. Built as part of the NVIDIA Holoscan Platform, DAQIRI uses DPDK to bypass the Linux kernel and move UDP and RoCE v2 traffic directly from the NIC into GPU DMA buffers at line rates exceeding hundreds of gigabits per second. The system automates packet reordering, flow steering, and on-GPU data conversions such as mapping wire int4 payloads into fp16 tensors using YAML configuration files and simple C++ or Python APIs. This enables real-time edge processing, filtering, and model inference across experimental pipelines like CERN ATLAS.


### [PP-OCRv6 on Hugging Face: 50-Language OCR from 1.5M to 34.5M Parameters](https://yomu.fyi/post/pp-ocrv6-on-hugging-face-50-language-ocr-from-1-5m-to-34-5m-parameters.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: AlexZhang, cuicheng, Jun Zhang, Manhui Lin, Yue Zhang, leo-q8, yubo, Yi Liu
- Published: Jun 22, 2026
- Type: Announcement

PP-OCRv6 is PaddleOCR’s latest universal OCR model family, targeting text detection and recognition across documents, screenshots, displays, labels, and scene text. It offers tiny, small, and medium tiers spanning 1.5M to 34.5M parameters, while the small and medium tiers support 50 languages. Its PPLCNetV4 backbone is shared across detection and recognition, with RepLKFPN for lightweight multi-scale detection and EncoderWithLightSVTR for recognition using local context and global attention. On PaddleOCR’s official in-house multi-scenario benchmarks, PP-OCRv6\_medium reaches 86.2% detection Hmean and 83.2% recognition accuracy, improving over PP-OCRv5\_server by 4.6 percentage points and 5.1 percentage points. PaddleOCR provides Paddle Inference, Transformers, and ONNX Runtime paths, and results can be saved as visualization images or structured JSON for downstream workflows.


### [We got local models to triage the OpenClaw repo for FREE!\*](https://yomu.fyi/post/we-got-local-models-to-triage-the-openclaw-repo-for-free.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Onur Solmaz, ben burtenshaw, shaun smith
- Published: Jun 22, 2026
- Type: Problem & solution

OpenClaw receives hundreds of issues and pull requests daily, creating a triage problem for maintainers who need near-real-time filtering without consuming a paid closed-model quota. The system uses Gemma and Qwen local models inside a Pi-based agent harness: each item is normalized, enriched with GitHub context, classified into finite topic labels, and required to return structured output. For pull requests, the agent can inspect the repository through reposhell, a read-only command-restricted shell, while localpager, SQLite, a worker queue, and deterministic notification policy handle orchestration and Discord delivery. Tests described for gemma-4-26b-a4b and qwen3.6-35b-a3b report hundreds of tokens per second and good zero-shot classification accuracy without fine-tuning, while an LLM audit loop monitors errors; the authors present the pattern as agentic classification for high-throughput triage.


### [Extending Stripe’s network with stablecoins](https://yomu.fyi/post/extending-stripe-s-network-with-stablecoins.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Sai Samant
- Published: Jun 22, 2026
- Type: Explainer

Stripe describes integrating stablecoin settlement into its existing Global Payments and Treasury Network (GPTN), rather than creating a parallel system. GPTN models accounts, wallets, and payment rails as a directed graph, allowing routing, netting, cross-entity settlement, FX, reconciliation, and liquidity management to span fiat and blockchain flows. For a payout, Stripe records a pending stablecoin obligation, Bridge draws from a stablecoin liquidity pool to credit the recipient within seconds to minutes, and fiat replenishment proceeds asynchronously over traditional rails; reverse flows work similarly. A direct mode bypasses the usual few-minute planning cycle for urgent wallet funding, trading netting efficiency for speed, while standard flows continue batching. The live infrastructure handles real traffic, with ongoing work focused on forecast-driven liquidity, faster replenishment through RTP and SEPA Instant, and broader product and corridor coverage.


### [Ten years of open source: a stained-glass view](https://yomu.fyi/post/ten-years-of-open-source-a-stained-glass-view.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jun 19, 2026
- Type: Explainer

ClickHouse’s ten-year open-source history is presented through a stained-glass analogy: many contributors remain visible as distinct pieces, while shared infrastructure holds the project together. The article connects that community account to production use through cBioPortal at Memorial Sloan Kettering, where moving heavy filtering into ClickHouse SQL made relevant paths about ten times faster, changing waits from minutes to seconds and supporting continued exploration. It also describes a demonstration built from bare-clone git history, one month of GH Archive events, and the shipped system.contributors table, all queried in ClickHouse. The account reports 100,000 issues and pull requests crossed on March 19, 2026, with 83% of issues and 99% of pull requests resolved at the time of the talk, and presents contributor credit and repair as operating parts of the project.


[Newer posts](https://yomu.fyi/topic/performance/page/14.md) · [Older posts](https://yomu.fyi/topic/performance/page/16.md)