---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [How controllers from industrial machinery can coordinate multitask machine learning](https://yomu.fyi/post/how-controllers-from-industrial-machinery-can-coordinate-multitask-mac.md)
- Company: amazon.science
- Author: Theodore Vasiloudis
- Published: Jul 30, 2026
- Type: Problem & solution

Multitask machine learning often suffers from gradient conflicts, drift, and drought when blending multiple objective gradients in single update steps. To address this in graph self-supervised learning, researchers developed ControlG, a framework that allocates discrete computational blocks to individual objectives over time. ControlG operates across three coupled loops: sensing objective difficulty via spectral demand and gradient interference, planning target compute allocations using log-hypervolume Pareto sensitivities, and executing the schedule via a proportional-integral-derivative controller. Evaluated across homophilic and heterophilic graph benchmarks, the approach outperforms traditional multitask gradient-manipulation baselines while incurring low computational overhead. The framework also produces interpretable training timelines and deficit traces.


### [NVIDIA Exemplar Cloud: Lessons for Unlocking Full Performance on AI Infrastructure](https://yomu.fyi/post/nvidia-exemplar-cloud-lessons-for-unlocking-full-performance-on-ai-inf.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 30, 2026
- Type: Problem & solution

Identical AI computing clusters can exhibit 8% to 12% training throughput deficits compared to NVIDIA reference architectures due to compounded configuration choices across kernels, hypervisors, BIOS, and NCCL settings. Profiling with Linux perf, turbostat, and NCCL tests uncovers performance degradation across multiple layers of the compute stack. In virtualized Grace deployments, enabling Arm SMMU Virtual Command Queue support eliminates invalidation serialization bottlenecks on memory-heavy workloads. On x86 systems, adjusting BIOS C-states to allow idle cores into C6 frees thermal headroom for active cores to achieve rated turbo frequencies, while cpuset isolation resolves NUMA contention. Further gains stem from tuning NCCL queue pairs per connection and properly bind-mounting topology files inside workload containers.


### [GPU Management: Why Idle GPUs Are the New Grounded Aircraft](https://yomu.fyi/post/gpu-management-why-idle-gpus-are-the-new-grounded-aircraft.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Erick Lachmann, Gabriel Pimenta de Freitas Cardoso, Gustavo Lucchetti
- Published: Jul 30, 2026
- Type: Explainer

Enterprise AI is shifting from a primary focus on model capability to the utilization of scarce, expensive GPU capacity. Like aircraft, GPUs accrue financing, depreciation, power, and cooling costs by calendar hour, while producing value only during useful compute hours; unlike aircraft, they cannot freely absorb workloads with different memory, latency, and duration profiles. The article explains how peak-sized clusters, mixed workloads such as training, fine-tuning, quantization, real-time and batch inference, embeddings, and evaluation can leave capacity stranded even when dashboards show high occupancy, and argues that API pricing can motivate enterprises to own infrastructure. It presents specialized models and GPU orchestration as complementary levers: specialization reduces each workload’s footprint, while management reallocates freed capacity across workloads and queues. The conclusion is that installed capacity alone does not determine returns; enterprises need both strategies to close the gap between GPUs owned and useful output.


### [How to use Google microbenchmarks for evaluating TPU performance](https://yomu.fyi/post/how-to-use-google-microbenchmarks-for-evaluating-tpu-performance.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Junjie Qian, Chi Shuen Lee, Yu-Hsuan (Amy) Lin, Haixiong (Sean) Wang
- Published: Jul 30, 2026
- Type: Explainer

Evaluating TPU performance requires measurements beyond product specifications, because real-world behavior depends on architecture and workload constraints. The suite measures network collectives, compute, HBM bandwidth, host transfers, and Ragged-Paged Attention primitives, using metrics such as GB/s, latency, TFLOPs, MFU, and TTFT-related throughput. Its src and configs directories provide implementations and tunable parameters, while sample Kubernetes recipes connect measurements to TPU architecture and runtime behavior. Roofline analysis uses peak MXU throughput and HBM bandwidth to distinguish compute-, memory-, and network-bound workloads and guide kernel, sharding, or rematerialization choices. In a 110B MoE workload on a 4x4x4 TPU 7x configuration, microbenchmark-guided changes addressed ICI stalls and memory-bound attention, reducing training step time by 21.2%.


### [ThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale](https://yomu.fyi/post/thunderagent-2x-faster-agentic-inference-for-synthetic-data-generation.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Jul 29, 2026
- Type: Announcement

ThunderAgent is a program-level scheduler for high-throughput agentic inference, developed by researchers at Georgia Tech, UIUC, CMU, and Together AI. It solves KV cache thrashing, which occurs when request-level engines like vLLM and SGLang evict an agent’s KV cache during tool calls and recompute it on resume, creating a cascade of evictions. ThunderAgent tracks each workflow as a schedulable program, pausing low-priority workflows under memory pressure and routing resumed programs through a global waiting queue to the node with the most available capacity. In synthetic data generation pipelines, it achieves up to 2.5x single-node throughput and 2.4x speedup on 8 nodes with near-linear scaling, and was accepted as an ICML 2026 Spotlight paper.


### [Building an agentic harness that outlasts the model](https://yomu.fyi/post/building-an-agentic-harness-that-outlasts-the-model.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jul 29, 2026
- Type: Explainer

Shopify’s Application Security team built an agentic harness that scans software for vulnerabilities, proves candidates with real tests, and opens Shopify-specific remediation pull requests. Its Dispatch orchestrator partitions repositories, runs category-specific Hunting agents in parallel, verifies findings sequentially with a different model, and persists architecture artifacts for later diff-based scans. Test oracles require realistic integration or functional coverage, including cross-tenant fixtures for IDOR checks, while unproven candidates are rejected or downgraded. Across more than 80 applications and thousands of scans, the workflow produced over 300 findings, including two rated Critical by Shopify’s severity calculator, with full scans costing $50–$300 and incremental scans $5–$50. The post concludes that durable value lies in the harness’s verification, partitioning, cross-model review, and deterministic infrastructure rather than in any single model.


### [Developing Healthcare Robotics with GPU-Native Medical Physics Simulation](https://yomu.fyi/post/developing-healthcare-robotics-with-gpu-native-medical-physics-simulat.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 28, 2026
- Type: Announcement

Healthcare robotics faces data scarcity, long-tail clinical scenarios, and multi-year development cycles due to reliance on physical phantoms and animal studies. To resolve this, NVIDIA Medical Physics Simulation introduces GPU-native simulation within NVIDIA Isaac for Healthcare, integrating classical physics solvers and generative world models on a single GPU. The Endoluminal Simulation Module models flexible instruments as Cosserat rods solved via globally coupled extended position-based dynamics, maintaining zero-copy PyTorch interoperability via NVIDIA Warp. Concurrently, Cosmos-H-Dreams leverages distilled video diffusion models via the FlashDreams runtime to stream action-conditioned surgical video at over 30 frames per second. This dual-track architecture enables synchronized physics, synthetic fluoroscopic rendering, and scaled reinforcement learning policies across hundreds of parallel GPU environments.


### [Indexing the Data Lake for Online Point Queries](https://yomu.fyi/post/indexing-the-data-lake-for-online-point-queries.md)
- Company: [Spotify](https://yomu.fyi/company/spotify.md)
- Author: Spotify Engineering
- Published: Jul 27, 2026
- Type: Problem & solution

Online services and AI agents require low-latency point queries over massive datasets that are economically impractical to store entirely in key-value systems like Bigtable or DynamoDB. While cloud object storage offers rapid request latency, traditional distributed query engines like Trino and BigQuery introduce seconds of planning and scheduling overhead along with dependent file reads. Random Access Parquet addresses this limitation by using an external index mapping keys directly to exact Parquet files and row locations. Readers execute O(1) index lookups and issue parallel ranged reads to fetch only the required bytes without scanning footers or metadata. By running directly against existing data lake files, this architecture eliminates separate storage copies and allows shared analytical data to serve interactive online lookups.


### [NVIDIA Ising Enables Fully Automated Quantum Computer Calibration with Enhanced In-Context Learning](https://yomu.fyi/post/nvidia-ising-enables-fully-automated-quantum-computer-calibration-with.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 27, 2026
- Type: Announcement

NVIDIA Ising Calibration is an open-source vision language model designed to interpret quantum processor diagnostic outputs and recommend tuning steps. The newly released Ising Calibration 1.5 features a 31-billion-parameter architecture that is 11.4% smaller at BF16 precision than its predecessor. Trained on multi-modality partner data including superconducting qubits, quantum dots, ions, neutral atoms, and electrons on Helium, the model supports both zero-shot reasoning and in-context learning. Evaluated on the QCalEval benchmark, it scores 10% higher on average than comparable open models in zero-shot tasks and improves in-context learning performance by over 86% relative to the prior release. In addition to BF16 weights, the model offers an NVFP4 quantization option suitable for deployment on single consumer GPUs or NVIDIA DGX Spark hardware.


### [NVIDIA Cosmos-H-Dreams: Bringing Real-Time Generative Simulation to Surgical Robotics](https://yomu.fyi/post/nvidia-cosmos-h-dreams-bringing-real-time-generative-simulation-to-sur.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Lukas Zbinden, Javier Gamazo, Mostafa Toloui, Sean Huver
- Published: Jul 27, 2026
- Type: Announcement

Surgical robotics needs safer, faster ways to train and evaluate vision-language-action policies because physical experiments are costly, slow to reproduce, and vulnerable to damaging instruments or biological material. NVIDIA introduces Cosmos-H-Dreams, a real-time, action-conditioned generative simulator that specializes Cosmos-H-Surgical-Simulator for da Vinci Research Kit tabletop suturing and generates future frames from an initial RGB image and live robot kinematics. Its causal student is trained through teacher fine-tuning, causal warmup, and self-forcing distillation, including failure episodes and long autoregressive horizons, while few-step diffusion reduces generation cost. Served through FlashDreams with streaming KV caching, CUDA Graph capture, and model compilation, it reaches approximately 160 frames per second on one NVIDIA RTX PRO 6000, compared with roughly 10 frames per second for standard inference. The system supports human or learned-policy control, but is presented as a research and development platform rather than a diagnostic system or physical-robot controller.


### [NVIDIA Nemotron 3 Ultra Leads Open Models on Accuracy and Efficiency in Agentic RTL Coding](https://yomu.fyi/post/nvidia-nemotron-3-ultra-leads-open-models-on-accuracy-and-efficiency-i.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Nirmal Kumar Juluru
- Published: Jul 27, 2026
- Type: Benchmark

Register transfer level (RTL) development relies heavily on iterative verification and tool feedback, which poses challenges for standard code generation models. To address realistic hardware design tasks, the ACE-RTL agent pairs an iterative generate-test-reflect workflow with NVIDIA Nemotron 3 Ultra. The model utilizes a hybrid Mamba-Attention Mixture-of-Experts architecture trained on a synthetic dataset encompassing code generation, editing, and error debugging. Across the comprehensive Verilog design problems benchmark, ACE-RTL with Nemotron 3 Ultra attained a 97.1% average pass rate, outperforming GLM 5.2 and Kimi K2.6. Additionally, Nemotron 3 Ultra achieved its accuracy while consuming an average of 6,629 tokens per iteration, which is substantially lower than competing models.


### [Advancing Semiconductor Innovation Across Materials Engineering and Manufacturing](https://yomu.fyi/post/advancing-semiconductor-innovation-across-materials-engineering-and-ma.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 27, 2026
- Type: Problem & solution

Increasing artificial intelligence workloads and slowing geometric scaling require rapid semiconductor innovation across materials engineering and manufacturing. Applied Materials collaborated with NVIDIA to build an end-to-end digital development pipeline that accelerates atomic-scale modeling, process development, and factory optimization. The Ginestra defect-centric simulation platform integrates NVIDIA cuDSS sparse solvers to achieve up to a 10x speedup, while NVIDIA cuEST accelerates density functional theory calculations on B200 systems by approximately 55x compared to 64 CPU cores. In process engineering, the ACE+ multiphysics platform pairs with NVIDIA PhysicsNeMo to run topography simulations up to 35x faster, and NVIDIA Omniverse generates fab digital twins to optimize material flows. This unified digital workflow enables engineers to perform virtual experiments rapidly, mitigate production bottlenecks, and shorten transitions to high-volume manufacturing.


### [ModelExpress: Distributing Model Artifacts at the Speed of Light](https://yomu.fyi/post/modelexpress-distributing-model-artifacts-at-the-speed-of-light.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 24, 2026
- Type: Problem & solution

Distributing large language model weights and JIT kernel caches across clusters imposes high latency on cold starts, autoscaling, and reinforcement learning loops. NVIDIA ModelExpress minimizes this delay by identifying existing compatible weight replicas across the cluster rather than treating every worker as an independent cold start. Initial replicas bootstrap by streaming directly from remote object storage into GPU memory or bypassing host staging using GPUDirect Storage. Subsequent replicas load pre-processed weights and kernel caches directly from serving peers over peer-to-peer RDMA via the NVIDIA Inference Xfer Library. In testing on DeepSeek-V4 Pro, this peer-to-peer artifact distribution transferred weights and caches in under ten seconds, reducing overall replica startup time from eight minutes to one minute and forty-four seconds.


### [Session Replay for Unreal Engine: see the crash before the crash](https://yomu.fyi/post/session-replay-for-unreal-engine-see-the-crash-before-the-crash.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Ivan Tustanivskyi
- Published: Jul 24, 2026
- Type: Announcement

Crash reports in non-deterministic game environments often lack player context, making callstack-only debugging difficult. Sentry's SDK version 1.16.0 for Unreal Engine introduces Session Replay to continuously record gameplay and attach the final video clip directly to crash reports. The cross-platform recording pipeline hooks into Slate's backbuffer presentation on the render thread, uses engine pixel shaders for format conversion, and feeds frames to hardware H.264 encoders via Unreal's AVCodecs plugins. Encoded fragments are organized in a rolling ring buffer and periodically written to temporary MP4 files that are uploaded alongside crash data. Benchmark results in a 4K demo game showed an overhead of approximately 0.20 ms per frame.


### [Debugging Ray Tracing Applications Using NVIDIA OptiX Toolkit](https://yomu.fyi/post/debugging-ray-tracing-applications-using-nvidia-optix-toolkit.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 23, 2026
- Type: Problem & solution

NVIDIA OptiX ray tracing applications often present difficult debugging challenges, including invalid API arguments, black frames, and concurrency issues across thousands of GPU threads. The open-source NVIDIA OptiX Toolkit addresses these diagnostic hurdles by offering unified error checking and targeted device-side debug logging. Host-side macros like OTK\_ERROR\_CHECK wrap OptiX, CUDA runtime, and CUDA driver API calls to enforce uniform error validation while capturing file and line diagnostics. For device-side troubleshooting, the toolkit introduces a DebugLocation utility to isolate logging to specific launch indices instead of flooding the output stream with concurrent thread data. This device mechanism also supports interactive one-shot debug dumps and screen highlights around the inspected pixel.


### [Bringing Nunchaku 4-bit Diffusion Inference to Diffusers](https://yomu.fyi/post/bringing-nunchaku-4-bit-diffusion-inference-to-diffusers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pham Hong Vinh, Sayak Paul
- Published: Jul 23, 2026
- Type: Announcement

Modern diffusion transformers running in BF16 often require 20–30 GB of VRAM, limiting access on consumer GPUs; the integration brings Nunchaku Lite checkpoints into Diffusers to reduce memory and denoising latency. SVDQuant handles activation outliers by moving them into weights, preserving a small 16-bit low-rank branch while quantizing residual weights to 4-bit, enabling W4A4 compute in transformer attention and MLP projections. Nunchaku Lite patches standard nn.Linear modules at runtime, loads Hub-hosted CUDA kernels through the kernels package, and exposes ordinary from\_pretrained() loading without local compilation; AWQ W4A16 handles modulation and normalization projections. On an RTX 5090, the cited NVFP4 checkpoint generates 1024×1024 images in about 1.7 seconds at roughly 12 GB peak memory versus about 24 GB for BF16, while the bare-bones Nunchaku Lite path delivers around 30% speedup; torch.compile raises reported end-to-end speedup from 1.35x to 1.8x.


### [Make Long-Running NVIDIA TensorRT Engine Builds Observable and Cancelable in Python or C++](https://yomu.fyi/post/make-long-running-nvidia-tensorrt-engine-builds-observable-and-cancela.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 22, 2026
- Type: Tutorial

Long-running NVIDIA TensorRT engine builds can freeze workflows and waste GPU hours when they lack visibility or abort mechanisms. Developers can resolve this by subclassing the IProgressMonitor API in Python or C++ and attaching it to the builder configuration. The monitor tracks hierarchical build phases through start, step completion, and finish callbacks, requiring thread synchronization because TensorRT calls monitor methods across internal builder threads. Returning false from the step completion callback requests early build cancellation, which unwinds active phases in reverse order at the next step boundary. Beyond terminal rendering with ANSI escape codes, teams can stream these progress events to IDE protocols, HTTP service endpoints, or agent tool-call traces.


### [.NET support for Godot is now generally available](https://yomu.fyi/post/net-support-for-godot-is-now-generally-available.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Serhii Snitsaruk
- Published: Jul 22, 2026
- Type: Announcement

Sentry released version 2.0.0 of its Godot Engine SDK, introducing general availability for C# and .NET error reporting alongside application metrics. While earlier 1.x releases captured errors from GDScript, native C and C++ code, and the engine itself, version 2.0.0 captures runtime exceptions from managed C# code on the same trace as engine and GDScript events. The SDK synchronizes scope elements such as tags, breadcrumbs, and user context, while integrating with source control managers like GitHub to fetch surrounding source code frames. Developers can also emit custom metrics over fixed intervals or game events to evaluate performance distributions and gameplay progression. These metrics allow slicing by custom attributes such as CPU and GPU hardware configurations to pinpoint hardware bottlenecks or rebalance game variables.


### [Session revocations at scale](https://yomu.fyi/post/session-revocations-at-scale.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Llew Vallis
- Published: Jul 22, 2026
- Type: Problem & solution

Canva gateways check in-memory session revocations on every request, but pulling over a million revocations from MySQL during deployments created severe database stampedes. To eliminate this bottleneck, the team transitioned to storing a twelve-hour sliding window of revocations in Amazon S3 partitioned into 30-minute chunks. Each revocation is packed into a 16-byte binary structure within a sorted flat array, allowing gateways to query records via direct binary search without object deserialization. Asynchronous workers batch updates into S3 using ZooKeeper leader election and conditional PUT requests for optimistic concurrency control. This architecture reduced the in-memory cache footprint by 87.5% and allowed the team to reduce their MySQL database to just two read replicas.


### [Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72](https://yomu.fyi/post/setting-a-world-record-for-moe-pre-training-on-nvidia-gb300-nvl72.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Kirthi Devleker
- Published: Jul 21, 2026
- Type: Benchmark

Frontier AI pre-training increasingly relies on mixture of experts architectures, which lower compute per token while introducing severe all-to-all communication bottlenecks. To address these scaling limits, the NVIDIA GB300 NVL72 rack-scale system co-designs Blackwell Ultra GPUs, fifth-generation NVLink scale-up fabrics, and 800 Gbps SuperNIC scale-out networking. Testing on DeepSeek-V3 671B across 256 GPUs achieved a pre-training benchmark of 1,648 TFLOPs per GPU using Megatron Core. In addition to hardware co-design, ongoing software optimizations across Megatron Core, TorchTitan, and JAX contributed major throughput gains over time. Scaling from 256 to 1,024 GPUs sustained over 97% per-GPU performance efficiency across all evaluated frameworks by overlapping scale-out gradient traffic behind compute windows.


[Newer posts](https://yomu.fyi/topic/performance/page/9.md) · [Older posts](https://yomu.fyi/topic/performance/page/11.md)