π0.7 (Physical Intelligence)¶
Session: Speaker(s): Paper: π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities, arXiv 2604.15483 (PDF) Blog: pi.website/blog/pi07 Code (earlier open models): Physical-Intelligence/openpi. It has π0, π0-FAST, and π0.5 in JAX and PyTorch, with base checkpoints pre-trained on more than 10k hours of robot data. π0.7 is not in it.
Paper summary (from the arXiv HTML and blog; figure-only numbers not yet read)¶
Team and date: Physical Intelligence, with about 100 authors including Sergey Levine, Chelsea Finn, and Karol Hausman. Published April 16, 2026; arXiv v2 is dated April 24, 2026. Availability: The weights are not open. Access goes through the waitlisted Pi API.
Architecture (about 5B parameters, confirmed in the paper)
- VLM backbone: Gemma3 4B, which includes a 400M vision encoder. It is supervised with FAST tokens under knowledge insulation: gradients from the action expert do not reach the VLM.
- Video history: A MEM-style encoder compresses history in time and space. Up to 4 cameras each provide up to 6 history frames at a 1 s stride.
- Action expert: An 860M flow-matching transformer. It predicts 50-step action chunks with 5 denoising steps and executes 15 or 25 of those steps.
- Proprioception: One linear-projected token for each history state, as in MEM. π0.6 encoded state as discretized text instead.
- Real-time chunking: Training simulates 0–12 timestep delays, which covers up to 240 ms of inference latency at 50 Hz.
- Control rate: 50 Hz on most robots and 20 Hz on the UR5e.
- Subgoal world model: A 14B mixture-of-transformers model based on BAGEL, trained with flow matching. It generates one subgoal image per camera at 448×448. It runs asynchronously, taking about 1.25 s for 25 steps (unverified).
- Inference latency (unverified): 38 ms minimal and 127 ms with MEM and subgoals, on one H100.
Prompt and context design
- Language: The task instruction, plus a subtask instruction that comes from a learned high-level policy or a human coach.
- Metadata fields:
- Speed: the episode length, binned to 500 steps.
- Quality: a score from 1 to 5.
- Mistake: true or false.
- Control mode: joint or end-effector.
- Example prompt:
Task: peel vegetables. Subtask: pick up the peeler. Speed: 8000. Quality: 5. Mistake: false. Control Mode: joint. - Visual subgoals: Multi-view images of a state up to 4 s ahead. They are refreshed when the subtask changes, or every 4 s.
- Dropout during training:
- Subgoal images are included on 25% of examples, and on those the subtask is dropped 30% of the time.
- All metadata is dropped 15% of the time, and each metadata field is dropped independently 5% of the time.
- Why mixed-quality data helps: Failures and autonomous rollouts are labeled with their actual speed, quality, and mistake values. The model learns to link those labels to behavior, so low-quality data stops contaminating the policy.
- At inference:
- Speed is set to the 15th percentile of episode length for that task, quality to 5, and mistake to false.
- Classifier-free guidance on the metadata uses β ∈ {1.3, 1.7, 2.2}.
Training data: Demonstrations from many robots (mobile and static bimanual, single-arm), in-house and in real homes. Also autonomous eval rollouts (including π*0.6 RL rollouts), human interventions, open-source robot datasets, egocentric human video, and web data. Hours are not stated.
Results
- Specialist tasks: On espresso, box building, and laundry, it matches or beats the RL-trained π*0.6 specialists, with higher throughput on laundry and box building. The exact numbers are only in figures.
- Zero-shot cross-embodiment shirt folding on a UR5e: π0.7 reaches 85.6% task progress and 80% success. Human teleoperators with an average of 375 hours of experience reach 90.9% and 80.6%. All the shirt-folding training data came from a different, smaller bimanual robot.
- Instruction following: It does much better than π0.5 and π0.6 on 14 multi-step scenarios in unseen kitchens and bedrooms. Subgoal images help further on referential instructions such as "pick up the object I would use to eat soup."
- Coaching: Step-by-step verbal coaching gets the model through new tasks such as loading an air fryer or toasting a bagel. A high-level policy fine-tuned on the coaching data then runs these tasks on its own.
- Ablations: With metadata removed, or with eval data removed, the model does much worse, especially on throughput. When laundry data is split into quality buckets, adding lower-quality data hurts without metadata and helps with it.
Limitations (from the model-extracted summary, unverified)
- Seen tasks often exceed 90% success, while unseen tasks reach about 60–80%.
- With so much training data, it is hard to say which tasks are truly unseen.
- Next steps the authors name: more detailed coaching and RL on the test tasks.
Talk notes¶
Metadata and mixed-quality data
- Metadata lets the model learn from data of mixed quality.
- Without metadata, adding lower-quality data hurts performance.
- With metadata, adding lower-quality data helps.
Emergent compositional generalization
- The result is a single generalist model.
- Example of an emergent capability: cross-embodiment generalization.
Day 4: the Do Robotics Need World Models? workshop
- A talk titled Long-Context, Long-Horizon, Compositional Generalization without World Models used π0.7 twice. See the Day 4 notes.
- As a limit: π0.7's context length is insufficient for long tracking tasks. The fix is compressed text memory, through memory narrations.
- As the generalization result: Detailed multimodal prompting lets the UR5e fold clothes with no folding data on that robot.
Takeaways for our work¶
- Diverse data plus rich multimodal prompts yields a single generalist model.