Start Customizing NVIDIA Nemotron 3 Nano with Prime Intellect Lab in Minutes

Summary

Adapting open models to specialized tasks often requires significant infrastructure, technical expertise, and GPU resources. This tutorial demonstrates how developers can customize NVIDIA Nemotron 3 Nano using Prime Intellect Lab for hosted reinforcement learning with verifiable rewards. The workflow establishes a baseline evaluation on a Python Math task, trains a LoRA adapter using a TOML configuration, and reevaluates performance. Across 32 held-out problems capped at five assistant turns, the model improves its accuracy from 21.9% to 90.6% for less than five dollars in compute spend. The same hosted training approach applies to larger models in the family, including Nemotron 3 Super and Nemotron 3 Ultra.

Context

Model customization typically requires specialized infrastructure, software, and domain knowledge to select algorithms and verify learning, making it challenging for developers to tailor models to specific tasks.

Approach / What changed

Using the Prime CLI, developers configure a Prime Intellect Lab workspace, evaluate baseline model performance in the math-python environment, and train a LoRA adapter for NVIDIA Nemotron 3 Nano using reinforcement learning with verifiable rewards before deploying and reevaluating the adapter.

Takeaways

• Baseline accuracy of Nemotron 3 Nano on 32 math-python test examples rose from 21.9% (7/32) to 90.6% (29/32) after reinforcement learning training.
• The experimental environment caps rollouts at five assistant turns, heavily penalizing models that repeatedly invoke tools without producing a final boxed answer.
• The same Prime Intellect Lab workflow scales to larger models, including Nemotron 3 Super and Nemotron 3 Ultra, by updating the model identifier in the configuration.

Related reading

• NVIDIA Developer Blog · Jul 1, 2026

  Mastering Agentic Techniques: AI Agent Reinforcement Learning

  Language model agents often fail across long-horizon workflows, repeat tool-call errors, and output invalid schemas when relying solely on prompting or retrieval. Reinforcement learning with verifiable rewards provides a practical training signal for domain-specific tasks by scoring model rollouts against deterministic verifiers such as schema checkers, unit tests, and execution sandboxes. Teams can implement group relative policy optimization to sample multiple trajectories per prompt and update policy weights based on relative performance within the group. Establishing environments with explicit harness, state, and verifier definitions allows agents to learn multi-step tool use, whereas static datasets fail to capture sequential dependencies. Inspecting checkpoint rollouts and testing against held-out tasks prevents reward hacking while continuously turning operational failures into verifiable benchmark environments.

  Elizabeth Goodman

• NVIDIA Developer Blog ·