Post-Train NVIDIA Cosmos 3 Edge for On-Device Robot Control

Summary

Deploying large world models on robots is constrained by onboard memory and the control latency needed for continuous manipulation. The tutorial post-trains NVIDIA Cosmos 3 Edge, a 4B omni-model with a 2B NVIDIA Nemotron-based reasoner, into a DROID action policy using 76,000 successful teleoperated trajectories, then serves it through an OpenPI-compatible WebSocket server on Jetson Thor. The configuration predicts 32 future 8-dimensional joint-position actions at 15 Hz, while a receding-horizon loop executes a prefix and replans from fresh camera and robot-state observations. On an NVIDIA Jetson AGX Thor T5000, inference takes about 1.53 seconds per action chunk covering roughly 2.13 seconds of motion, and the policy reaches 22.9% success in closed-loop RoboLab evaluation; the source contrasts this with Cosmos 3 Nano’s 36.8%.

Takeaways

1. 1

   Cosmos 3 Edge is a 4B omni-model with a 2B NVIDIA Nemotron-based reasoner, and its weights occupy roughly 9 GB in BF16 for Jetson Thor deployment.

2. 2

   The policy emits 32 future 8-dimensional joint-position actions at 15 Hz; the example loop executes the first 16 actions before requesting a new chunk.

3. 3

   The post-trained Edge policy reaches 22.9% success across the closed-loop RoboLab task suite, compared with 36.8% for Cosmos 3 Nano, reflecting the stated trade-off for fully on-robot real-time operation.