Hydra-0: Action Flow for Generalist World Modeling and Control¶
Session: NVIDIA's Hybrid Simulation: Cosmos Neural with Isaac Lab talk on Day 3. Slide photos taken 11:02–11:06 am. See the Day 3 notes. Paper: Technical report, preprint, August 2026. The slide showed an arXiv button but no ID, and code is "coming soon." Authors: Hongyu Li, Bowen Wen, Xinghao Zhu, Yixuan Wang, Yilun Du, Yunzhu Li, George Konidaris, Stan Birchfield, Soha Pouya, Chenran Li, Yan Chang (NVIDIA, Brown, Columbia, Harvard). Project page: nvidia-isaac.github.io/video_to_data/hydra-0. The URL suggests it belongs to NVIDIA's Video to Data effort.
Paper summary (from a slide photo of the project page's abstract; the paper itself is not yet read)¶
- The idea: A generalist world model conditioned on action flow, which represents robot actions as pixel motion.
- Hybrid simulator: At deployment, a physics engine moves the robot and a learned video model predicts how the world responds.
- What one model does: It trains across embodiments, simulates them, evaluates policies, and drives a real robot.
Talk notes¶
- Object flow as the shared action interface: A System 2 agent plans the desired object motion as flow, and a System 1 policy turns that flow into robot actions.
- Demo: Given one image and "Drag the purple cloth towards the left," the agent proposed a 29-point flow on the cloth.