Loading post…
NVIDIAMichelle HortonAnnouncement
NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning
Summary
NVIDIA released Nemotron 3.5 Super VL, an open 120B-parameter multimodal model for specialized enterprise agents. Its mixture-of-experts architecture activates 12B parameters per token, balancing capacity and inference efficiency. The model combines reasoning, instruction following, tool use, and understanding of text, images, documents, and video for tasks including coding, web research, document intelligence, and cybersecurity. On a single GPU, it achieves up to 5x higher throughput for agentic tasks than competing open models, and it scores higher than the larger Nemotron 3 Ultra on Hermes Bench and LangChain evaluations. Early adopters report cost reductions of 40x for credit processing and 10x lower latency for receipt parsing, and organizations can post-train the model with NVIDIA NeMo for domain-specific workflows and deploy it with NVIDIA NIM and Dynamo, which serves up to 8x more users with 85% lower first-token latency.
Context
Always-on agents are taking on longer, more complex enterprise workflows. A single request can involve reading a document, checking information across sources, following company policy, and calling several tools.
Approach / What changed
The model uses a hybrid Mamba-Transformer mixture-of-experts architecture with 120B total parameters and 12B active per token. It was distilled from Nemotron 3 Ultra with SkewReverse loss for coding accuracy. Post-training uses NVIDIA NeMo for supervised fine-tuning, low-rank adaptation, and reinforcement learning. Inference optimizations include speculative decoding (up to 2.3x faster generation), NVFP4 mixed-precision quantization via quantization-aware distillation, multi-token prediction, and prefix caching for context reuse. Deployment leverages NVIDIA NIM and Dynamo, which supports KV-cache-aware routing and disaggregated serving.
Takeaways
Nemotron 3.5 Super VL activates only 12B of its 120B parameters per token, balancing capacity with inference efficiency and achieving up to 5x higher throughput on a single GPU versus comparable open models.
The model can be post-trained with NVIDIA NeMo using domain-specific data, tools, and policies; early adopters report cost reductions of 40x and latency reductions of 5x to 10x on specialized workloads like credit processing and receipt parsing.
Inference optimizations include speculative decoding (up to 2.3x faster generation), NVFP4 quantization via quantization-aware distillation, multi-token prediction, and prefix caching, enabling efficient always-on agent deployment with NVIDIA Dynamo serving up to 8x more users with 85% lower first-token latency.
FAQ
What inference optimizations does Nemotron 3.5 Super VL use for agentic tasks?
It uses speculative decoding (up to 2.3x faster generation), multi-token prediction, prefix caching for context reuse across agent turns, and NVFP4 mixed-precision quantization via quantization-aware distillation.
How can developers customize Nemotron 3.5 Super VL for their domain?
Through NVIDIA NeMo open libraries: NeMo AutoModel for supervised fine-tuning and low-rank adaptation, and NeMo Gym for reinforcement learning in agentic environments. Open datasets on Hugging Face supplement proprietary data.
What performance gains do early adopters report?
Scale AI achieved 40x lower inference cost and 5x lower latency for credit processing. Ramp expects 50% higher throughput and 10x lower latency for receipt parsing. AI21 and Alice improved threat intelligence accuracy by 41%.
Source and freshness
- Source
- developer.nvidia.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.