---
title: "Beyond VLAs: How World Action Models Reshape Robot Manipulation"
description: "Robot policies built on vision-language-action models often struggle with physical generalization because vision-language backbones describe scenes instead of predicting physical dynamics. World action models address this limitation by building policies on video world models that jointly predict actions and future world states. NVIDIA Cosmos 3 provides an open foundation using a Mixture-of-Transformers architecture where an autoregressive transformer guides a diffusion transformer to synthesize continuous modalities including video and robot actions. Specializing Cosmos 3 on the DROID platform produced policies capable of simultaneously outputting action trajectories and predicted video observations while boosting RoboLab success from 28.1% to 36.8%. These models support flexible deployment tiers, from workstation serving on RTX PRO 6000 GPUs to real-time 15 Hz on-device execution on NVIDIA Jetson Thor embedded hardware."
---

# Beyond VLAs: How World Action Models Reshape Robot Manipulation

[NVIDIA](https://yomu.fyi/company/nvidia-developer-blog) · Michelle Horton · Aug 4, 2026

**Type:** Explainer

## Summary

Robot policies built on vision-language-action models often struggle with physical generalization because vision-language backbones describe scenes instead of predicting physical dynamics. World action models address this limitation by building policies on video world models that jointly predict actions and future world states. NVIDIA Cosmos 3 provides an open foundation using a Mixture-of-Transformers architecture where an autoregressive transformer guides a diffusion transformer to synthesize continuous modalities including video and robot actions. Specializing Cosmos 3 on the DROID platform produced policies capable of simultaneously outputting action trajectories and predicted video observations while boosting RoboLab success from 28.1% to 36.8%. These models support flexible deployment tiers, from workstation serving on RTX PRO 6000 GPUs to real-time 15 Hz on-device execution on NVIDIA Jetson Thor embedded hardware.

## Context

Standard vision-language-action (VLA) models add action modules to vision-language backbones that describe scenes semantically rather than predicting physical world dynamics. Consequently, VLA-based robot policies require near-identical demonstration data and struggle to generalize physically when object shapes, positions, or lighting conditions change.

## Approach / What changed

World action models (WAMs) build robot policies on video world models that jointly predict actions and future video states to leverage physical priors. NVIDIA Cosmos 3 implements this as an open omni-model using a Mixture-of-Transformers architecture, combining an autoregressive transformer for discrete text tokens with a diffusion transformer that iteratively denoises continuous modalities such as video, audio, and actions.

## Takeaways

- Cosmos 3 utilizes a Mixture-of-Transformers architecture that couples an autoregressive transformer for text with a diffusion transformer to iteratively denoise continuous modalities like video and actions.
- Starting post-training from a Cosmos 3 omni checkpoint trained on multi-domain action data improved RoboLab task success from 28.1% to 36.8% over a base checkpoint under identical compute and data.
- The 4B Cosmos 3 Edge policy operates on-device on NVIDIA Jetson Thor hardware, processing 640x360 camera inputs and generating 32 actions per inference to achieve real-time 15 Hz control.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source)

- Source: [NVIDIA](https://developer.nvidia.com/blog/beyond-vlas-how-world-action-models-reshape-robot-manipulation)
- Source URL: https://developer.nvidia.com/blog/beyond-vlas-how-world-action-models-reshape-robot-manipulation
- Ingested by Yomu: 2026-08-28T04:01:02.688Z

[Read original post](https://developer.nvidia.com/blog/beyond-vlas-how-world-action-models-reshape-robot-manipulation)