How to Train a Cross-Embodiment Robot Navigation Policy with AI Agents

Summary

The tutorial presents an agent-driven workflow for adapting robot navigation across embodiments and scenes, addressing the repeated cost of collecting data, preparing simulation assets, training, diagnosing, and evaluating each robot-scene pair. Using Boston Dynamics Spot, it applies COMPASS to a built-in warehouse and a SAGE-10K scene, with NVIDIA Omniverse NuRec described as an option for captured environments. COMPASS reuses the pretrained NVIDIA X-Mobility navigation policy and trains a residual reinforcement-learning specialist that corrects base actions for a selected robot and environment; specialists can later be distilled into a shared policy. Repository skills guide dependency checks, asset preparation, smoke testing, training, checkpoint comparison, and packaging, while human gates control scene acceptance, smoke-test continuation, and checkpoint promotion. Runtime integration converts camera images, goals or routes, and speed from odometry into velocity commands, with cuVSLAM available separately when compatible odometry and transforms are unavailable.

Takeaways

1. 1

   COMPASS reuses navigation behavior from the pretrained NVIDIA X-Mobility policy and trains a residual reinforcement-learning specialist for a selected robot and environment instead of relearning navigation from the beginning.

2. 2

   The workflow requires reviewable evidence across validation, scene preparation, training, evaluation, and packaging; human approval gates determine whether the process continues or a checkpoint is promoted.

3. 3

   cuVSLAM is a separate deployment component for camera-based odometry when compatible robot odometry and transforms are unavailable; its map is not an input to the navigation policy.