---
title: "NVIDIA"
description: "38 posts about NVIDIA, summarised, each linking to the original."
---

# NVIDIA
> 38 posts about NVIDIA, summarised, each linking to the original.

## Articles

### [How NVIDIA DSX MaxLPS Maximizes AI Factory Throughput and Efficiency](https://yomu.fyi/post/how-nvidia-dsx-maxlps-maximizes-ai-factory-throughput-and-efficiency.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Sarah McKenney
- Published: Sep 28, 2026
- Type: Benchmark

AI factories often underutilize power capacity due to static provisioning for peak GPU draw. NVIDIA DSX MaxLPS uses policy-governed dynamic power allocation to reclaim stranded capacity. In a joint evaluation with Nscale using Kimi K2.5 workloads on GB300 NVL72 systems, DSX MaxLPS enabled 37% more GPUs (192 vs 140) within the same 264.4 kW budget. Aggregate throughput increased 49.2% to 1.62 million tokens/s, while per-instance throughput remained unchanged and median latency stayed within 5% of baseline. However, P99 time to first token increased 17%, highlighting the need to evaluate tail latency alongside capacity gains.


### [Efficient MoE Training for Biological Foundation Models](https://yomu.fyi/post/efficient-moe-training-for-biological-foundation-models.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Sep 24, 2026
- Type: Tutorial

Mixture-of-experts (MoE) architectures scale model capacity more efficiently than dense transformers by activating only a subset of experts per token, but naive implementations suffer from fragmented expert computation, routing overhead, and memory pressure. NVIDIA Transformer Engine (TE) addresses these bottlenecks with optimized primitives including GroupedLinear for batched expert GEMMs, MXFP8 low-precision training, and a fused GroupedMLP kernel that combines quantization, SwiGLU activation, and routing-weight scaling. The BioNeMo recipe implements these techniques for biological foundation models, using TE’s autocast API for MXFP8 precision and the Sequential API to fuse the gate-up, ScaledSwiGLU, and down projections into a single operation. On eight NVIDIA B200 GPUs, the recipe achieved up to 2.21x the training throughput of the Hugging Face baseline for a Mixtral-8x7B model. The tutorial provides prerequisites, code examples, and configuration commands to reproduce the optimizations.


### [Validate GPU Cluster Readiness Before AI Workloads Land](https://yomu.fyi/post/validate-gpu-cluster-readiness-before-ai-workloads-land.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Sep 23, 2026
- Type: Explainer

GPU clusters can pass standard health checks yet fail under real AI training workloads due to a single degraded GPU, network link, or misrouted traffic. NVIDIA Cluster Readiness Engine (NVCRE) is an open source Kubernetes controller that runs real distributed workloads across topology-aware node groups before production jobs land, measuring results and reporting which nodes failed each test. Its layered API uses Custom Resource Definitions for Certification, Workflow, and Job resources, attributing each failure to a specific node and category such as NCCL bandwidth or training goodput. The WorkloadRun API handles platform detection, NCCL settings, shared-memory volumes, and gang scheduling, so operators do not need to write those manifests by hand. NVCRE integrates with NVSentinel for passive telemetry and is available under Apache 2.0, with a roadmap for inference and automated lifecycle validation.


### [How to Use NVIDIA Warp and MjWarp to Accelerate Robotics Simulation and Learning Workflows](https://yomu.fyi/post/how-to-use-nvidia-warp-and-mjwarp-to-accelerate-robotics-simulation-an.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Johnny Nuñez Cano, Asier Arranz, Rishabh Chadha, Ben Oliveri
- Published: Sep 23, 2026
- Type: Tutorial

NVIDIA Warp is a Python framework for writing GPU-accelerated kernels that compiles statically typed Python to CUDA. MuJoCo Warp (MJWarp) implements MuJoCo physics on Warp, enabling batched simulation of hundreds to thousands of independent robot environments on NVIDIA GPUs. The core API transition is small: mjw.put\_model and mjw.put\_data replace mujoco.MjModel and mujoco.MjData, and mjw.step advances the entire batch. Proper throughput measurement requires GPU synchronization and warm-up to avoid timing queue latency instead of actual GPU work. The post demonstrates moving an SO-101 follower arm from a single CPU world to 2,048 parallel MJWarp environments, validating parity and measuring aggregate world-steps per second.


### [Manage Kubernetes Node Fleets with NodeWright](https://yomu.fyi/post/manage-kubernetes-node-fleets-with-nodewright.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Sep 23, 2026
- Type: Problem & solution

Managing Kubernetes node fleets, especially GPU nodes, is difficult because host-level changes can disrupt long-running training jobs, and traditional tools are not cluster-aware. NodeWright, an open-source Kubernetes-native package manager from NVIDIA, solves this by declaratively configuring and updating node operating systems while respecting PodDisruptionBudgets and non-interruptible workload labels. The operator orchestrates a sequence of cordon, wait, drain, apply, interrupt, and uncordon on each node, using container image packages that carry modifications such as kernel parameters, security agents, and verification scripts. It supports progressive rollouts across a fleet via DeploymentPolicy compartments with configurable disruption budgets and strategies like batch, canary, and rolling. NodeWright integrates with NVIDIA AI Cluster Runtime for validated configurations and is part of the DSX OS open-source portfolio, available under Apache 2.0.


### [\*\*Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization\*\*](https://yomu.fyi/post/know-who-spoke-when-build-real-time-multi-speaker-ai-with-nvidia-nemot.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Francesco, Ivan Medennikov, Taejin Park, Jagadeesh Balam, Adi- margolin, Maryam Motamedi
- Published: Sep 23, 2026
- Type: Announcement

NVIDIA Nemotron 3 Diarization is an open-weight, 100M-parameter model that ranks #1 on VoiceArena’s Diarization-Bench leaderboard with a 14.72% Diarization Error Rate (DER). It classifies who spoke when in conversations, handling overlapping speech and supporting up to eight speakers across live and recorded audio. The model uses a Sortformer approach with arrival-order speaker labeling, a 31-layer Transformer encoder, and two memory mechanisms for streaming inference. It accepts 16 kHz single-channel audio and outputs a floating-point tensor of speaker-activity probabilities at a configurable time step. Developers can combine its timestamps with automatic speech recognition to produce speaker-attributed transcripts, with recommended input-buffer latencies ranging from 0.32 to 30.4 seconds.


### [Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing](https://yomu.fyi/post/enabling-private-high-performance-production-ai-inference-with-nvidia.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 22, 2026
- Type: Explainer

NVIDIA Confidential Computing (CC) enables secure AI inference by using memory-encrypted confidential VMs, confidential GPUs, and encrypted NVLink. Running inference frameworks like TensorRT LLM in a CC environment introduces performance overhead because secure execution changes assumptions about memory movement, timing, and multi-GPU communication. To quantify this overhead, the post presents a controlled methodology comparing CC-on and CC-off performance on a workload with long input context, extended output generation, and low concurrency. With CC-aware adaptations—including pageable memory selection, asynchronous readback, GPU %globaltimer for autotuning, and CC-aware multi-GPU communication—TensorRT LLM retained 96.1–98.2% of baseline throughput on eight NVIDIA B200 GPUs while keeping per-token latency overhead below 5%. The post concludes that security configuration and inference optimization should be treated as a single deployment problem for organizations moving private inference into production.


### [Topology-Aware Workload Scheduling with NVIDIA Topograph](https://yomu.fyi/post/topology-aware-workload-scheduling-with-nvidia-topograph.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 22, 2026
- Type: Explainer

AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.


### [Accelerating a ROS 2 Node with an AI Agent and NVIDIA Isaac ROS](https://yomu.fyi/post/accelerating-a-ros-2-node-with-an-ai-agent-and-nvidia-isaac-ros.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 22, 2026
- Type: Tutorial

GPU acceleration alone does not guarantee fast ROS 2 graphs because messages may still be serialized or copied through CPU memory. NVIDIA contributed the rosidl::Buffer abstraction and CUDA buffer backend to ROS 2 Lyrical, enabling zero-copy transport of GPU-resident payloads between co-located nodes when they share the same host, CUDA device, Linux user, and a supported RMW implementation, with automatic CPU fallback otherwise. An AI coding agent using the migrate-node-to-rosidl-buffer skill can audit an existing CUDA-accelerated node by tracing data movement, running a copy-boundary audit, and implementing a minimal interface-preserving patch. The tutorial demonstrates this on the Depth Anything 3 TensorRT node, updating subscription options and message storage to use CUDA buffer backend without changing the sensor\_msgs/msg/Image message definition. The resulting accelerated workload can be deployed on NVIDIA Jetson AGX Thor, preserving standard ROS 2 messages while avoiding unnecessary serialization and host copies.


### [Making Local AI Smarter and Faster](https://yomu.fyi/post/making-local-ai-smarter-and-faster.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Vladimir Sotnikov
- Published: Sep 18, 2026
- Type: Announcement

Junie Local’s update introduces Qwen3.8-3.6-27B-blend, a merged model that balances speed and capability for local coding agents. The blend, created by merging Qwen3.6 and Qwen3.8 weights in equal proportions, completed 37 tasks in a 100-task internal benchmark, compared to 34 for Qwen3.6 and 39 for Qwen3.8, while generating 71% fewer output tokens than Qwen3.8. The runtime uses multi-token prediction (MTP), with a two-token proposal on M5 MacBook Pro making decoding 60% faster. A four-bit MTP head was kept over an eight-bit one due to similar acceptance rates and memory savings. The update also brings experimental NVIDIA RTX support on Windows for machines with at least 24 GB of VRAM.


### [TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor](https://yomu.fyi/post/tensorrt-edge-llm-completes-the-mlperf-edge-agentic-benchmark-6-4x-fas.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 16, 2026
- Type: Benchmark

Edge agentic workloads require repeated tool-calling turns, growing conversation histories, and valid function calls on constrained devices rather than single-prompt responses. On one NVIDIA Jetson AGX Thor Developer Kit, TensorRT Edge-LLM ran Qwen3.6-27B in SingleStream mode and completed the benchmark’s 1,007-turn performance workload in 24 minutes and 36 seconds, versus 2 hours and 37 minutes for the llama.cpp reference run. It reported 52.33 tokens per second, 247.12 ms median time to first token, and 87.94% BFCL overall accuracy. The implementation combines NVFP4 weights and activations, FP8 KV cache, reusable prompt prefixes with recurrent-state restoration, and tree-based multi-token prediction; roughly 96% of prompt tokens used hot cache, while tree-based MTP could achieve about 40% more decoding performance than linear MTP with three draft steps.


### [How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin](https://yomu.fyi/post/how-nvidia-groq-3-lpx-deterministic-execution-drives-power-efficient-h.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 15, 2026
- Type: Explainer

AI factories are constrained by site power, making performance per watt especially important for high-interactivity inference. The NVIDIA Groq 3 LPX rack uses deterministic execution: its compiler schedules computation, data movement, and clock-cycle timing across 256 LPU chips, then derives a cycle-level current-demand curve. Preemptive Power (PEP) prepares the power-delivery network for upcoming spikes, while Clock Period Synthesis (CPS) lengthens or shortens selected cycles to reduce di/dt, voltage droop, and the voltage guardband. Internal testing reported more than 60% less voltage drop, while the source estimates a high-single-digit baseline-voltage reduction and potentially low-double-digit workload-power reduction; paired with Vera Rubin NVL72, the platform is described as delivering up to 35x higher throughput per megawatt than GB200 NVL72 for 2T+ parameter models at long context and high interactivity.


### [How NVIDIA NVLink 6 Delivers Multi-Layer Resiliency for AI Factories](https://yomu.fyi/post/how-nvidia-nvlink-6-delivers-multi-layer-resiliency-for-ai-factories.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 15, 2026
- Type: Explainer

NVIDIA NVLink 6 is presented as a scale-up networking fabric for large AI factories, where dropped packets, link degradation, and node interruptions can reduce training goodput or inference capacity. It connects 72 Rubin GPUs in the Vera Rubin NVL72 into a single scale-up domain and uses a multi-layer resiliency stack spanning physical, link, application, software, and system layers. Physical Layer techniques include lightweight FEC, Physical Layer Retry (PLR), UPHY recovery, and hardware replay buffers, while credit-based flow control (CBFC) prevents injection when the next hop lacks buffer capacity. Redundant switch trays and management paths, local fault containment, NMX high availability, checkpointing, topology-aware recovery, and targeted switch maintenance are described as mechanisms for containing failures and preserving workloads; the post concludes that this integrated design targets continuous operation and maximum cluster productivity.


### [Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine](https://yomu.fyi/post/accelerating-dropless-moe-training-in-jax-with-nvidia-transformer-engi.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Sep 14, 2026
- Type: Explainer

Production-scale mixture-of-experts (MoE) training must handle uneven, data-dependent expert loads, creating ragged tensors and expensive dispatch, combine, and all-to-all communication. Dropless MoE processes every selected token, preserving model quality but requiring kernels that support variable token counts without padding or dropping. The post presents JAX and NVIDIA Transformer Engine optimizations centered on group-aware MXFP8 quantization, grouped GEMM, and fused expert-parallel (EP) dispatch and combine using NCCL EP. Grouped GEMM computes each expert’s actual workload in one kernel path, while EP reduces movement overhead and can deduplicate tokens crossing networks. For DeepSeek-V3 training on NVIDIA GB200, the reported throughput increased from 103 to 1,068 TFLOPS/GPU, a 10.4x improvement; the post also reports roughly 10x throughput improvement and 97% scaling efficiency at 1,024 GPUs.


### [Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo](https://yomu.fyi/post/building-pinterest-s-vlm-serving-stack-on-nvidia-dynamo.md)
- Company: [Pinterest](https://yomu.fyi/company/pinterest.md)
- Author: Pinterest Engineering
- Published: Sep 10, 2026
- Type: Explainer

Pinterest describes building a VLM serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo for multimodal products including Pinterest Assistant, hybrid search, multimodal reranking, content understanding, signal generation, and content safety. Unlike text-only workloads, requests may contain thousands of images, costly and variable prefill, long multi-turn histories, and large KV caches, making preprocessing, routing, memory, and latency central serving concerns. The implementation combines Dynamo's distributed orchestration, multimodal frontend and vLLM support, KV-aware routing, disaggregated encoder/prefill/decode serving, and LMCache-based KV offloading with vLLM, an OpenAI Chat Completions interface, model-aware routing, EKS, and custom deployment components. Production-oriented AIPerf DAG benchmarks model QPS patterns, multi-turn conversations, many-image requests, and agentic DAGs; the team concludes that payload design, cache-aware routing, offload tiers, and a shared platform are essential to meeting product targets and accelerating new launches.


### [Microsoft, NVIDIA, and Anthropic partnerships](https://yomu.fyi/post/microsoft-nvidia-and-anthropic-partnerships.md)
- Company: [Anthropic](https://yomu.fyi/company/anthropic.md)
- Published: Sep 10, 2026
- Type: Announcement

Microsoft, NVIDIA, and Anthropic announced strategic partnerships to scale Anthropic’s Claude models on Microsoft Azure and broaden enterprise access to them. Anthropic committed to purchase $30 billion of Azure compute capacity and contract additional capacity up to one gigawatt, initially using NVIDIA Grace Blackwell and Vera Rubin systems. NVIDIA and Anthropic will collaborate on design and engineering to optimize Anthropic models for performance, efficiency, and total cost of ownership, while adapting future NVIDIA architectures to Anthropic workloads. Microsoft Foundry customers will be able to access Claude Sonnet 4.5, Claude Opus 4.1, and Claude Haiku 4.5, while Claude remains available across Microsoft’s Copilot products. Microsoft and NVIDIA committed up to $5 billion and $10 billion, respectively, to Anthropic; Amazon remains its primary cloud provider and training partner.


### [How Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra](https://yomu.fyi/post/how-full-stack-nim-optimizations-deliver-2-5x-more-users-on-nemotron-3.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Sep 10, 2026
- Type: Benchmark

Production serving for agentic AI must maximize concurrent users on fixed GPU infrastructure while preserving latency, especially with long prompts, reused context, and streamed responses. The article presents NVIDIA NIM as a deployable microservice that packages validated model- and GPU-aware choices, then measures Nemotron 3 Ultra NIM 2.0.12 against an open-source baseline on four B200 GPUs using a 64K/400 workload, 76% KV reuse, and 50 TPS per user (20 ms ITL). The optimized stack combines autotuned MoE and Mamba kernels, tensor and expert-aware parallelism, prefix and Mamba state-cache reuse, scheduling, batching, memory tuning, and MTP speculative decoding. At the stated target, throughput rises from 718 to 1,997 output tokens per second—more than 2.5x—while the article recommends replaying representative traffic with NVIDIA AIPerf to select a Pareto point meeting the application’s latency constraint.


### [To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!](https://yomu.fyi/post/to-infinity-and-beyond-thunderkittens-now-on-nvidia-vera-rubin-nvl72.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Sep 10, 2026
- Type: Explainer

Together’s kernels team ported ThunderKittens to NVIDIA Vera Rubin NVL72 and investigated why existing Blackwell GEMMs underuse the platform. The work targets NVFP4 and FP8 GEMMs: although Rubin preserves Blackwell’s programming model, the original kernels reach only about 42.1% and 44.4% of roofline because tensor cores consume operands twice as fast as the old pipeline supplies them. The team integrates Rubin’s 64-byte K step, exclusive tensor memory extending to 576 columns, oversized shared memory up to 328 KiB, B-side collector buffers, and earlier A-release signaling into its NVFP4 kernel. The resulting kernel exceeds 22 PFLOPS and is described as competitive with cuBLAS and CuTe DSL, while the post also compares Rubin’s specifications with NVIDIA HGX B200.


[Newer posts](https://yomu.fyi/topic/nvidia.md)