NVIDIA introduces Cosmos 3 models for robotics

NVIDIA has released Cosmos 3, an open omni-model world foundation model built on a Mixture-of-Transformers architecture designed to transition robotics from vision-language-action models to World Action Models. Trained on a dataset of 767M images, 348M videos of real-world dynamics, and 8M action samples, Cosmos 3 is available in three sizes: 4B NVIDIA Cosmos Edge, 16B NVIDIA Cosmos Nano, and 64B NVIDIA Cosmos 3 Super. This foundation allows robots to predict physical dynamics and scene evolution rather than just mapping language to actions.

To demonstrate this capability, NVIDIA developed specialized policies for the DROID platform, which features a Franka Panda arm and a Robotiq gripper. The 16B Cosmos3-Nano-Policy-DROID and the 4B Cosmos3-Edge-Policy-DROID generate action trajectories alongside predicted video of the outcome. In testing, starting with an omni checkpoint trained on multi-domain action data raised RoboLab success from 28.1% to 36.8% compared to a base checkpoint, proving the value of physical priors.

For practitioners, Cosmos 3 offers flexible deployment tiers. The 16B Nano policy can run on a workstation equipped with a single NVIDIA RTX PRO 6000, streaming actions to the robot over a network. For on-device edge computing, the 4B Edge policy runs directly on hardware like the NVIDIA Jetson Thor, Jetson T2000, and Jetson T3000 modules. It processes 640x360 observations to generate 32 actions per inference, achieving real-time control at 15 Hz.

This shift to World Action Models fundamentally changes how roboticists train their systems. Instead of requiring massive, highly specific demonstration datasets to teach basic physics to a language-based model, developers can leverage the pre-existing physical priors of Cosmos 3. Because the model understands how objects slide, fall, or bend, it generalizes to unseen environments and new robot embodiments with significantly fewer demonstrations, drastically reducing the cost and time of data collection.