How to Build on BlueField with DOCA Agent Skills
A practical how-to for using NVIDIA DOCA Agent Skills to generate BlueField code with verified APIs and hardware checks, reducing correction cycles and failed builds.
Loading…
Topic
NVIDIA
32 posts about NVIDIA. Every summary links to the original.
- 32 posts
- 6 companies
- 2 explainers
See the idea
NVIDIA explainers
How BioNeMo Turns a Target List Into Folded Structures
BioNeMo Inference Runtime runs one pipeline per target: parse, tokenize, generate features, infer on a GPU replica, and write a structure. This diagram traces those stages and the throughput trade-off.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Enabling Private High-Performance Production AI Inference with NVIDIA Confidential Computing
NVIDIA Confidential Computing (CC) enables secure AI inference by using memory-encrypted confidential VMs, confidential GPUs, and encrypted NVLink. Running inference frameworks like TensorRT LLM in a CC environment introduces performance overhead because secure execution changes assumptions about memory movement, timing, and multi-GPU communication. To quantify this overhead, the post presents a controlled methodology comparing CC-on and CC-off performance on a workload with long input context, extended output generation, and low concurrency. With CC-aware adaptations—including pageable memory selection, asynchronous readback, GPU %globaltimer for autotuning, and CC-aware multi-GPU communication—TensorRT LLM retained 96.1–98.2% of baseline throughput on eight NVIDIA B200 GPUs while keeping per-token latency overhead below 5%. The post concludes that security configuration and inference optimization should be treated as a single deployment problem for organizations moving private inference into production.
Topology-Aware Workload Scheduling with NVIDIA Topograph
AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.
Accelerating a ROS 2 Node with an AI Agent and NVIDIA Isaac ROS
GPU acceleration alone does not guarantee fast ROS 2 graphs because messages may still be serialized or copied through CPU memory. NVIDIA contributed the rosidl::Buffer abstraction and CUDA buffer backend to ROS 2 Lyrical, enabling zero-copy transport of GPU-resident payloads between co-located nodes when they share the same host, CUDA device, Linux user, and a supported RMW implementation, with automatic CPU fallback otherwise. An AI coding agent using the migrate-node-to-rosidl-buffer skill can audit an existing CUDA-accelerated node by tracing data movement, running a copy-boundary audit, and implementing a minimal interface-preserving patch. The tutorial demonstrates this on the Depth Anything 3 TensorRT node, updating subscription options and message storage to use CUDA buffer backend without changing the sensor_msgs/msg/Image message definition. The resulting accelerated workload can be deployed on NVIDIA Jetson AGX Thor, preserving standard ROS 2 messages while avoiding unnecessary serialization and host copies.
Making Local AI Smarter and Faster
Junie Local’s update introduces Qwen3.8-3.6-27B-blend, a merged model that balances speed and capability for local coding agents. The blend, created by merging Qwen3.6 and Qwen3.8 weights in equal proportions, completed 37 tasks in a 100-task internal benchmark, compared to 34 for Qwen3.6 and 39 for Qwen3.8, while generating 71% fewer output tokens than Qwen3.8. The runtime uses multi-token prediction (MTP), with a two-token proposal on M5 MacBook Pro making decoding 60% faster. A four-bit MTP head was kept over an eight-bit one due to similar acceptance rates and memory savings. The update also brings experimental NVIDIA RTX support on Windows for machines with at least 24 GB of VRAM.
TensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor
Edge agentic workloads require repeated tool-calling turns, growing conversation histories, and valid function calls on constrained devices rather than single-prompt responses. On one NVIDIA Jetson AGX Thor Developer Kit, TensorRT Edge-LLM ran Qwen3.6-27B in SingleStream mode and completed the benchmark’s 1,007-turn performance workload in 24 minutes and 36 seconds, versus 2 hours and 37 minutes for the llama.cpp reference run. It reported 52.33 tokens per second, 247.12 ms median time to first token, and 87.94% BFCL overall accuracy. The implementation combines NVFP4 weights and activations, FP8 KV cache, reusable prompt prefixes with recurrent-state restoration, and tree-based multi-token prediction; roughly 96% of prompt tokens used hot cache, while tree-based MTP could achieve about 40% more decoding performance than linear MTP with three draft steps.
How NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin
AI factories are constrained by site power, making performance per watt especially important for high-interactivity inference. The NVIDIA Groq 3 LPX rack uses deterministic execution: its compiler schedules computation, data movement, and clock-cycle timing across 256 LPU chips, then derives a cycle-level current-demand curve. Preemptive Power (PEP) prepares the power-delivery network for upcoming spikes, while Clock Period Synthesis (CPS) lengthens or shortens selected cycles to reduce di/dt, voltage droop, and the voltage guardband. Internal testing reported more than 60% less voltage drop, while the source estimates a high-single-digit baseline-voltage reduction and potentially low-double-digit workload-power reduction; paired with Vera Rubin NVL72, the platform is described as delivering up to 35x higher throughput per megawatt than GB200 NVL72 for 2T+ parameter models at long context and high interactivity.
How NVIDIA NVLink 6 Delivers Multi-Layer Resiliency for AI Factories
NVIDIA NVLink 6 is presented as a scale-up networking fabric for large AI factories, where dropped packets, link degradation, and node interruptions can reduce training goodput or inference capacity. It connects 72 Rubin GPUs in the Vera Rubin NVL72 into a single scale-up domain and uses a multi-layer resiliency stack spanning physical, link, application, software, and system layers. Physical Layer techniques include lightweight FEC, Physical Layer Retry (PLR), UPHY recovery, and hardware replay buffers, while credit-based flow control (CBFC) prevents injection when the next hop lacks buffer capacity. Redundant switch trays and management paths, local fault containment, NMX high availability, checkpointing, topology-aware recovery, and targeted switch maintenance are described as mechanisms for containing failures and preserving workloads; the post concludes that this integrated design targets continuous operation and maximum cluster productivity.
Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine
Production-scale mixture-of-experts (MoE) training must handle uneven, data-dependent expert loads, creating ragged tensors and expensive dispatch, combine, and all-to-all communication. Dropless MoE processes every selected token, preserving model quality but requiring kernels that support variable token counts without padding or dropping. The post presents JAX and NVIDIA Transformer Engine optimizations centered on group-aware MXFP8 quantization, grouped GEMM, and fused expert-parallel (EP) dispatch and combine using NCCL EP. Grouped GEMM computes each expert’s actual workload in one kernel path, while EP reduces movement overhead and can deduplicate tokens crossing networks. For DeepSeek-V3 training on NVIDIA GB200, the reported throughput increased from 103 to 1,068 TFLOPS/GPU, a 10.4x improvement; the post also reports roughly 10x throughput improvement and 97% scaling efficiency at 1,024 GPUs.
Building Pinterest’s VLM Serving Stack on NVIDIA Dynamo
Pinterest describes building a VLM serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo for multimodal products including Pinterest Assistant, hybrid search, multimodal reranking, content understanding, signal generation, and content safety. Unlike text-only workloads, requests may contain thousands of images, costly and variable prefill, long multi-turn histories, and large KV caches, making preprocessing, routing, memory, and latency central serving concerns. The implementation combines Dynamo's distributed orchestration, multimodal frontend and vLLM support, KV-aware routing, disaggregated encoder/prefill/decode serving, and LMCache-based KV offloading with vLLM, an OpenAI Chat Completions interface, model-aware routing, EKS, and custom deployment components. Production-oriented AIPerf DAG benchmarks model QPS patterns, multi-turn conversations, many-image requests, and agentic DAGs; the team concludes that payload design, cache-aware routing, offload tiers, and a shared platform are essential to meeting product targets and accelerating new launches.
Microsoft, NVIDIA, and Anthropic partnerships
Microsoft, NVIDIA, and Anthropic announced strategic partnerships to scale Anthropic’s Claude models on Microsoft Azure and broaden enterprise access to them. Anthropic committed to purchase $30 billion of Azure compute capacity and contract additional capacity up to one gigawatt, initially using NVIDIA Grace Blackwell and Vera Rubin systems. NVIDIA and Anthropic will collaborate on design and engineering to optimize Anthropic models for performance, efficiency, and total cost of ownership, while adapting future NVIDIA architectures to Anthropic workloads. Microsoft Foundry customers will be able to access Claude Sonnet 4.5, Claude Opus 4.1, and Claude Haiku 4.5, while Claude remains available across Microsoft’s Copilot products. Microsoft and NVIDIA committed up to $5 billion and $10 billion, respectively, to Anthropic; Amazon remains its primary cloud provider and training partner.
How Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra
Production serving for agentic AI must maximize concurrent users on fixed GPU infrastructure while preserving latency, especially with long prompts, reused context, and streamed responses. The article presents NVIDIA NIM as a deployable microservice that packages validated model- and GPU-aware choices, then measures Nemotron 3 Ultra NIM 2.0.12 against an open-source baseline on four B200 GPUs using a 64K/400 workload, 76% KV reuse, and 50 TPS per user (20 ms ITL). The optimized stack combines autotuned MoE and Mamba kernels, tensor and expert-aware parallelism, prefix and Mamba state-cache reuse, scheduling, batching, memory tuning, and MTP speculative decoding. At the stated target, throughput rises from 718 to 1,997 output tokens per second—more than 2.5x—while the article recommends replaying representative traffic with NVIDIA AIPerf to select a Pareto point meeting the application’s latency constraint.
To Infinity and Beyond: ThunderKittens Now on NVIDIA Vera Rubin NVL72!
Together’s kernels team ported ThunderKittens to NVIDIA Vera Rubin NVL72 and investigated why existing Blackwell GEMMs underuse the platform. The work targets NVFP4 and FP8 GEMMs: although Rubin preserves Blackwell’s programming model, the original kernels reach only about 42.1% and 44.4% of roofline because tensor cores consume operands twice as fast as the old pipeline supplies them. The team integrates Rubin’s 64-byte K step, exclusive tensor memory extending to 576 columns, oversized shared memory up to 328 KiB, B-side collector buffers, and earlier A-release signaling into its NVFP4 kernel. The resulting kernel exceeds 22 PFLOPS and is described as competitive with cuBLAS and CuTe DSL, while the post also compares Rubin’s specifications with NVIDIA HGX B200.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.