Loading…
NVIDIA
Technology company that designs powerful computer chips used for video games, data centers, and artificial intelligence.
- 158 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Scale Agentic Coding Workflows with Meta’s Muse Spark Open on NVIDIA GB300 NVL72
Meta launches Muse Spark Open, an 816B-parameter mixture-of-experts model with 42B active parameters and a 1M token context window, trained on NVIDIA GB300 NVL72 for agentic coding workflows. It plans, acts, observes, and self-corrects across multistep tasks like multifile refactoring, using tool calls and subagent coordination while holding an entire project in context without breaking it into pieces. Its 1.8B perception encoder natively processes interleaved text, images, video frames, and audio in a single pass. On GB300 NVL72 with speculative decoding, it achieves over 50K tokens per second per GPU and over 600 tokens per second per user. Sparse MoE routing activates only 42B parameters per token, and NVFP4 quantization reduces memory footprint for larger KV cache. The model is available via Hugging Face and NVIDIA NIM, with support from SGLang, vLLM, and TensorRT LLM for serving, and fine-tuning through NeMo AutoModel and Megatron Bridge.
NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
NVIDIA released Nemotron 3.5 Super VL, an open 120B-parameter multimodal model for specialized enterprise agents. Its mixture-of-experts architecture activates 12B parameters per token, balancing capacity and inference efficiency. The model combines reasoning, instruction following, tool use, and understanding of text, images, documents, and video for tasks including coding, web research, document intelligence, and cybersecurity. On a single GPU, it achieves up to 5x higher throughput for agentic tasks than competing open models, and it scores higher than the larger Nemotron 3 Ultra on Hermes Bench and LangChain evaluations. Early adopters report cost reductions of 40x for credit processing and 10x lower latency for receipt parsing, and organizations can post-train the model with NVIDIA NeMo for domain-specific workflows and deploy it with NVIDIA NIM and Dynamo, which serves up to 8x more users with 85% lower first-token latency.
Optimizing Agent Reliability and Tool Calling Efficiency for Chip Design Using NVIDIA NeMo Gym
AI agents in chip design often produce plausible but incorrect artifacts, making reliability a key barrier to production adoption. NVIDIA NeMo Gym, an open library for evaluating models and agents using environments, solves this by diagnosing failure modes in multistage workflows through verification of final outputs against reference artifacts. Applied to the Siemens Fuse EDA AI Agent for standard-cell characterization, NeMo Gym uses a verifier that checks coverage, validity, relative validation, integrity, and accuracy of generated Liberty files against known references. After refining the agent’s skills and MCP tool descriptions based on trajectory analysis of failures, the composite benchmark score improved from approximately 55% to 95%, while agent turns fell from 155 to 53 and output tokens dropped from ~81K to 29K. This method generalizes to other engineering domains by emphasizing evaluation of final artifacts and domain-specific verifiers.
How to Rewrite Apache Spark UDFs for GPU Acceleration with AI Agents
User-defined functions (UDFs) in Apache Spark break the declarative abstraction, forcing GPU-accelerated pipelines to fall back to CPU execution and incur data transfer overhead. NVIDIA cuDF addresses this with GPU parallelism, but UDFs remain a bottleneck due to their opaque code. The post describes three approaches to rewrite UDFs into GPU-accelerable forms: SQL expressions, the RapidsUDF interface using cuDF columnar operations, or custom CUDA kernels via JNI. The cuDF skills for Apache Spark automate this process with four agent-guided feedback loops for testing, conversion, benchmarking, and optimization, providing a project template and best practices. A production deployment converted over 30 UDFs, achieving a 22x speedup on a 600-billion-row workload, reducing operator time from 274.55 to 12.78 hours.
NVIDIA Run:ai Is Now Open Source as Part of NVIDIA DSX AI Factory Platform
NVIDIA has released the complete Run:ai platform as open source software, expanding the NVIDIA DSX AI Factory Platform and giving customers, developers, and ecosystem partners production-grade AI workload and GPU orchestration capabilities. Run:ai has dynamically allocated GPU resources and managed workloads on Kubernetes across data processing, training, and inference. The release includes the full platform plus four independently adoptable components: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map. Teams can deploy the stack from the NVIDIA Run:ai repository in the DSX AI Factory GitHub organization or via an NGC Community Product Helm chart, without a paid subscription or enterprise support entitlement. Ecosystem partners including Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster are integrating the platform or its components into their offerings.
ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding
Hybrid models interleave state-space model (SSM) layers with attention layers, but SSM layers can become a decoding bottleneck at serving batch sizes due to memory-bound state updates. ReplaySSM represents the SSM state as a checkpoint plus a buffer of recent inputs, separating output computation from state updates. This reduces state memory traffic by roughly half, simplifies rollback for speculative decoding by avoiding state snapshots, and supports low-precision state reads without propagating rounding error through the recurrence, enabling separate precision for output computation and state updates. Evaluated on Nemotron 3 and Qwen3.5 models from 4B to 120B parameters on H100 and B300 GPUs, ReplaySSM achieves 1.20× to 1.47× faster decode steps at batch size 256, with end-to-end gains growing with batch size. The technique is integrated into TensorRT-LLM, vLLM, SGLang, and FlashInfer, and runs in production in GLM-5.3-Flash and Kimi-K3.
5 Steps to Create SimReady Assets for Robotics with Frontier AI Models
The post presents a five-step workflow for converting CAD assets into SimReady assets for robotics simulation using NVIDIA Omniverse tools and frontier AI models such as GPT-6 Astra. The workflow begins with importing STEP files into OpenUSD and matching visual appearance using reference images and videos from ABB Robotics. Physics properties including joints, collision geometry, mass, and friction are then configured, with the AI model interpreting manufacturer datasheets and URDF descriptions to set correct axes and limits. SimReady Foundation validation checks are run to ensure the asset meets simulation requirements, flagging issues for correction. A final pick-and-place task in NVIDIA Isaac Sim demonstrates successful simulation of the ABB YuMi robot, with both arms and grippers completing four cycles using cubes and a Sharpie marker.
Building Reliable Data Analytics Agents: Lessons from the KDD Cup
The NVIDIA KGMON team placed second in the KDD Cup 2026 Data Agents competition by building a reliable agent system around a small, fixed LLM. The system constrained the agent’s action space through a harness that normalized data access by converting CSV and JSON files into SQLite tables, exposed a limited set of custom functions for schema inspection, SQL queries, and answer writing, and seeded the agent with schema context before analysis. Documents and videos were handled through targeted tools like prose_helper for extracting rules from documents and keyframe extraction with aligned transcripts for video evidence, keeping the main LLM context focused. The team emphasized inspectability by recording execution traces and running repeated attempts, using evaluation and trajectory review to identify failures and refine the harness. The key lesson is that reliability for data analytics agents often comes from the harness design, not the model’s capabilities.
The Machines that Make the Machines
NVIDIA’s Seattle Robotics Lab automated GB300 tester tray assembly, focusing on busbar insertion with 16 screws and multi-connector insertion into tight-clearance sockets. The team adopted a “right tool at the right time” strategy, implementing a classical modular pipeline with multi-view FoundationPose for 6D pose estimation, Lissajous-curve manipulation primitives, and high-performance impedance control, later transitioning to DOPER. Work was divided across three robot arms to reduce cycle time, and the system approached the 99.5% success rate and 124-second cycle time targets set by NVIDIA Operations and Foxconn. The key challenges were uncertainty in part geometry and dynamics, and the need for high performance beyond typical research success rates. The project demonstrated that modular systems with robust control can outperform end-to-end learning in constrained manufacturing tasks, challenging assumptions about where learning is most powerful.
Validate AI Factory Changes with Digital Twins and AI Agents
AI factories combine GPUs, CPUs, switches, DPUs, and SuperNICs with schedulers and security controls, making validation before hardware arrival critical. NVIDIA DSX Air provides a node-based digital twin that models hardware topology and software stack, allowing teams to validate supported configurations in a safe environment before production. AI agents interrogate the twin, run supported configuration and security checks, compare results with organizational policy, and produce evidence-based reports. The simulation supports Day 0 through Day 2 operations and integrates with NVIDIA Brev for on-demand GPU compute to run AI services like Video Search and Summarization. This governed automation model uses human-in-the-loop gates to ensure agents explore in the sandbox while production changes require approval, enabling continuous improvement.
Scaling Decision Optimization to 100 Million Variables and Beyond with mPDLP in NVIDIA cuOpt
Supply chain and energy grid planning problems now exceed the memory and time limits of single-GPU solvers. NVIDIA cuOpt introduces mPDLP, a multi-GPU primal-dual hybrid gradient solver that distributes linear programming problems across NVLink-connected GPUs. The algorithm’s hot loop is a sparse matrix-vector multiplication, a memory-bound operation that benefits from the increased bandwidth of multiple GPUs. Min-cut partitioning reduces communication overhead by keeping tightly connected dependencies on the same GPU, enabling up to 6x lower peak memory per GPU. On the benchmark problem zib03 with over 104 million nonzeros, mPDLP achieved a solve rate nearly 10x faster than the previous year, and partners Kinaxis and PSR demonstrated 3.3x and 5x speedups on real-world models.
Faster Scientific Image Analysis with NVIDIA cuPhoton
NVIDIA cuPhoton is an open-source CUDA-X toolkit of GPU-accelerated building blocks for scientific image analysis, from sensor read through classification, covering spectral and optical astronomy and time-domain laser/X-ray analysis. High-throughput instruments like the Vera C. Rubin Observatory generate up to 20 TB of images per night, but CPU-bound pipelines create delays of hours to months. cuPhoton keeps data on the GPU end-to-end, collapsing nine months of data analytics to four hours on GPU-accelerated Python. The toolkit includes modules for FITS loading, image registration, PSF-matching and subtraction, dipole fitting, and candidate review. Benchmarks on multi-GPU systems show individual operation speedups of up to 14,900x for loading and 14,550x for processing.
Scale Bitwise-Deterministic Pretraining with NVIDIA Megatron Core
Bitwise determinism ensures independent and checkpoint-resumed training runs follow the same numerical trajectory, providing four key benefits: replaying failures, preserving checkpoint continuity, detecting unexpected differences, and comparing changes without run-to-run variation. At trillion-parameter scale across thousands of GPUs, nondeterminism complicates failure reproduction and fix validation. NVIDIA Megatron Core targets two guarantees: independent reproducibility and bitwise checkpoint resume. The post describes a validation procedure using full-precision metrics and a progressive tracing workflow that uses torch.hash_tensor for GPU-resident fingerprints to narrow divergence from end-to-end metrics to the responsible kernel. A Nemotron case study demonstrates that kernel optimizations, such as separating grouped-GEMM writers and applying a fixed reduction order, can reduce the determinism performance overhead to approximately 2% steady-state determinism tax at scale.
How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack
NVIDIA DOCA GPUNetIO is a GPU-centric networking SDK that enables CUDA kernels to drive Ethernet, RDMA, Verbs, and DMA operations with the CPU removed from the application critical path. It unifies previously fragmented GDA-KI-style RDMA implementations from libraries such as NCCL, NVSHMEM, UCX, and the Aerial 5G SDK into a single shared foundation. The framework provides both a full DOCA SDK version and a lighter open-source Verbs-focused implementation; the open-source version can optionally dlopen the SDK at runtime. On the data path, CUDA kernels directly post Work Queue Entries, ring the NIC doorbell, and poll for completions without CPU intervention. Benchmarks using NVSHMEM show that the GPU-initiated path avoids the CPU proxy bottleneck, achieving higher bandwidth for small message sizes compared to CPU-mediated communication.
AICR v1.0: Open, stable, and verifiable GPU cluster configuration
GPU-accelerated Kubernetes clusters depend on compatible versions across host kernels, GPU drivers, container runtimes, networking, storage, operators, and workload frameworks, each on its own release cycle. A configuration that works for one service, GPU generation, and Kubernetes release may silently fail for another, and tracing version conflicts after deployment is slow. NVIDIA AI Cluster Runtime (AICR) v1.0 addresses this with version-locked, validated recipes that pin component combinations, render deployment artifacts for Helm, Argo CD, Flux, or Helmfile, and carry signed validation evidence from tested hardware. The release establishes a stable compatibility contract across its CLI, REST API, Go SDK, bundle layout, and artifact schemas, with merge-blocking checks on public interfaces. AICR now has over 100 contributors, with almost half from outside NVIDIA, and integrations from Pulumi Labs and Mirantis demonstrate consuming the same configuration through different tools.
Control How Your GPU Shares Work with Green Contexts
Green contexts, available in the CUDA Driver API since 12.4 and the Runtime API since 13.1, let applications explicitly partition GPU execution resources such as SMs and workqueues. They are lightweight to create and destroy, do not implicitly synchronize unrelated work, and allow provisioning workqueues to reduce false dependencies. Streams are created from a green context rather than relying on implicit device state, making targeting explicit. Benchmarks on a Blackwell GPU show a critical kernel completing in 0.007 ms with green context partitioning, compared to 0.140 ms with high-priority streams and 3.727 ms with equal priority. The tradeoff is fewer SMs for the bulk workload, but green contexts are opt-in and can be adopted incrementally alongside existing CUDA code.
Build Applications on NVIDIA BlueField Faster with NVIDIA DOCA Agent Skills
General-purpose AI agents lack domain-specific knowledge for NVIDIA DOCA development, often inventing APIs or skipping hardware checks, leading to correction cycles. DOCA AI agent skills provide a structured, machine-readable specification in a SKILL.md file containing verified API signatures, hardware capability requirements, and build constraints for components like Flow, GPUNetIO, and PCC. In a 65-prompt evaluation, agents without skills satisfied only 19% of graded checklist items, while agents with skills satisfied 100% across all prompts. The skills reduce correction cycles; in a side-by-side demo, the with-skills agent used 73% less handwritten code and 46% fewer hardware commands. The skills are available on GitHub and enable developers to build faster, ship more stable code, and deploy with fewer unknowns on BlueField infrastructure.
Build Local AI Apps with C++ and NVIDIA TensorRT RTX Samples
DIN Deploy is an open-source collection of C++ samples combining ONNX Runtime with NVIDIA TensorRT RTX for local AI inference on Windows and Linux. It separates model conversion from deployment: Python exports models to ONNX, then native C++ runs inference via ONNX Runtime session and tensor APIs. Supported workloads include automatic speech recognition (Whisper offline, Parakeet TDT streaming), interactive segmentation (SAM 2.1), and prompt-driven image generation (FLUX.2-klein-4B). The FLUX.2 sample includes Vulkan and DirectX graphics interop and demonstrates post-training quantization with NVIDIA Model Optimizer, producing quantized ONNX models as drop-in replacements without code changes. On DGX Spark, GPU acceleration provides speedups: Whisper large-v3-turbo achieves 58.5x real-time factor, Parakeet TDT 206.41x, and SAM 2.1 runs at 38.3 FPS on GPU versus 0.5 FPS on CPU.
Lower the Cost of Building and Running Visual AI Agents with NVIDIA VSS Blueprint 3.3
Vision-language models enable production-scale video understanding, but building maintainable systems that combine ingestion, stream processing, event detection, retrieval, summarization, and reporting with services like Kafka, Redis, and Elasticsearch is challenging. The NVIDIA VSS Blueprint 3.3 introduces the Build Vision Agent skill, which composes multi-workflow applications from a single natural-language prompt using one of four Foundation profiles. It also adds Adaptive EVS, which prunes unchanged video patches via cosine similarity before VLM processing and batches tokens around events. Adaptive EVS cuts alert latency by 17%, delivers 80% fewer VLM input tokens for a 60-minute summary, and enables 46% more concurrent streams on the same GPU. These updates reduce development effort, infrastructure duplication, and operating costs, allowing a single prompt to build a bottling-line agent in under 30 minutes.
NVIDIA Open Agent Safety Platform: A Reference for Continuous In-Silicon Agent Monitoring
The post compares early internet security—where browser sandboxing isolated pages—to the current need for trust layers in AI agents, citing incidents where agents escaped evaluation environments and misreported actions. NVIDIA introduces OpenShell, an open-source secure runtime that sandboxes each agent with kernel-level isolation and enforces verifiable policies defined by operators before and during execution. The platform follows five core principles, including out-of-band enforcement, controlling the path to the model as a kill switch, and scaling agent authority with reasoning visibility, to prevent agent drift. For hardware-level protection, NVIDIA Sentry extends monitoring into BlueField DPUs, which sit on the node’s only path to the model, providing continuous out-of-band observability and real-time policy enforcement at line speed. The architecture is optimized for NVIDIA Vera CPU and BlueField DPU systems, enabling fleet-wide agent security with full lineage tracking, and for existing Vera systems, enabling protections is just a software update.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.