Day 2: Tue, Nov 10, 2026¶
Keynotes¶
- Russ Tedrake (MIT; CEO, Walden Robotics), 3:30 pm CT: Learning in the Factories. Title and time come from the @corl_conf keynote post. TechTimes says it covers factory-trained large behavior models, but that is TechTimes' description, not the conference's. The official keynotes page lists no times, so check the on-site schedule.
Sessions attended¶
-
Boston Dynamics: Single Action Model. Session and speaker not yet recorded. The "Final thoughts" slide photo was taken at 1:56 pm. I found no public paper, blog post, or program listing for it as of Nov 10. Boston Dynamics is a diamond-level CoRL 2026 sponsor, so this may have been a sponsor or industry talk.
-
Franka Robotics: Franka Research 3 and FCI. Speaker and session not yet recorded. Slide photos taken at 1:58 and 2:02 pm. Notes under Live notes.
- Meta Robotics Studio (Jonathan Thompson, Reality Labs): home humanoid. Session and time not yet recorded. Notes under Live notes.
- AGIBOT: Scaling Robot Intelligence, Building a Physical AI Ecosystem. Speaker, session, and time not yet recorded. Notes under Live notes.
- Tesla (Patrick Chao): Building Robot Foundation Models. Session and time not yet recorded. Notes under Live notes.
Papers covered¶
| Paper | Group | Headline | Status | Links | Weights |
|---|---|---|---|---|---|
| DAVATaR | UT Austin | Fluidic tactile and immersion-depth sensing plus glare and zoom augmentation; 71% vs 41% mean success without tactile | summarized | page | ? |
| TAM | Ai2, KAIST | Sim-only residual torque layer below any policy; real Panda box pushing 47.6% to 76.2%, BC flip 50% to 72% | summarized | arXiv · page · code | ✅ |
Day 2 themes¶
Cross-day versions: deployment, human data, world models.
- Deployment is the product: Tedrake's 33,000 camshafts and 340.3 s between interventions, and Boston Dynamics' "data quality before scale." Lab results aren't enough.
- Data pyramids: Meta (teleop, then UMI, then egocentric video) and Tedrake (deployment data will dwarf everything else) both rank data sources. They disagree on what sits on top in the long run.
- World models as simulators: Tesla wants a causal, interactive, physically accurate world simulator for evaluation and RL. AGIBOT's GO-3 puts reasoning and future prediction in front of the action.
- Low-level layers trained in sim: TAM's residual torque layer works under any policy, and DAVATaR adds tactile sensing for hard perception conditions.
Live notes¶
Boston Dynamics: Single Action Model¶
- Final thoughts: data quality before scale (from slide photo, 1:56 pm):
- "For whole-body manipulation and for very dexterous manipulation, RL is hard to beat."
- The photos show Atlas at a workcell and Atlas's hands, in a lab setting.
- Follow-up Boston Dynamics talks on Thursday, Nov 12 (workshops), from the same slide:
- Yeuhi Abe and Robin Deits: Our Journey from MPC to RL. 9:50 am, Everything Beneath the Policy workshop.
-
Michael Lutter: Sim-2-Real RL for Manipulation. 2:55 pm, Sim 2 Real 2 Field workshop.
-
Diagrams: Lutter's Day 4 talk expands on both points, with diagrams for the fleet loop and demo resets.
Submerged manipulation: DAVATaR (UT Austin)¶
Speaker and session not recorded. The paper is from UT Austin (Chin and Martín-Martín labs), not Boston Dynamics.
- Problem: Submerged tasks are hard. The air–water interface distorts what the camera sees:
- Refraction: Objects appear shifted and at the wrong scale.
- Reflection: Glare from the surface hides the scene.
- Foam: Bubbles and foam on the surface block the view.
- Paper: DAVATaR: Depth-Aware Visually Augmented Tactile Learning Across the Air-Water Interface. These notes match the paper's Fig. 1. Full summary in papers/davatar.md.
Torque adaptation: TAM (Ai2, KAIST)¶
Speaker and session not recorded. The project page lists it as a Spotlight.
- TAM is trained entirely in simulation. It's a residual-torque layer under the low-level controller, so the same weights work under RL, BC, and MPC policies. Full summary in papers/tam.md.
Keynote: Russ Tedrake, Learning from the Factories¶
The slide title reads "Learning from the factories". The @corl_conf post says "Learning in the Factories".
- Framing: From the research lab, to deployment, to real customer value.
- The next hurdle: Turning general-purpose robots into a product that returns real value to customers.
- Walden Robotics 101:
- Founded in January 2026 (from slide photo).
- Walden's approach:
- Build the company to deploy and iterate very fast.
- Work from a set of core beliefs. Beliefs held when Walden started (from later in the talk):
- Base models will get better fast. Plan for it.
- Mid-training data is being commoditized, and stronger base models need less of it.
- Deployment data is still very valuable. Even a PhD-level manipulator still needs job training in the factory. Stronger base models actually make deployment data more valuable, not less.
- The technology is ready. What's left is to scale it and keep advancing the AI. He has high conviction that it will work.
- The most valuable data is in the field.
-
"It's very reasonable to be skeptical" (from slide photo): "In the limit, deployed robot data will dwarf everything else."
- Today's pyramid, from top to bottom: teleop; then ego videos, sim, UMI, gloves, and general robot data; then language and vision at the base.
- Deployment flips it: an inverted funnel with deployment data at the top. Below it sit ego videos, sim, UMI, gloves, and general robot data, then multimodal video, world modelling, language, and reasoning.
-
Manufacturing first, then everything else.
- Learning from manufacturing experts: Many of the problems robotics needs to solve have been solved before, with different technology and under different constraints.
- Deployment story: 33,000 camshafts and counting. This is Walden's first and longest-running deployment.
- The customer sets the requirements.
- Deploying is more expensive. The extra costs:
- Field ops and logistics.
- Contracts, relationships, and trust. "You have to earn it."
- With a fixed budget for internal robot data: Training distribution ≈ test distribution. My reading, not his words: data collected in deployment matches the test distribution by construction, which is the case for paying the extra cost.
- Core development vs the deployment flywheel. The tension:
- Deploying needs 97–99% success.
- So the task has to be constrained.
- So the task saturates.
- So there's no novelty left to carry into the next task.
- Initial HG-DAgger scaling laws. HG-DAgger is human-gated DAgger: an operator takes over when the policy goes wrong, and those interventions become training data.
- Question: How does stronger pretraining help in deployment?
- Finding: Stronger pretrained models need fewer interventions, yet still improve more from them.
- The interventions are also qualitatively different.
- Experiment setup (from slide photo):
- Data mixture: robot demonstrations, egocentric human video, and UMI.
- Pretraining scales: 1k, 10k, and 100k hours of external data, all on the same 4.7B architecture. These are the generalist models.
- Task demonstrations: 10 hours per task, which turn each generalist into a task-adapted model.
- Deployment: Intervention demonstrations collected during deployment feed later model updates.
-
Plot: mean time between interventions (MTBI), log scale, over 6 deployment weeks (from slide photos; values at the last point shown):
Model Weeks shown MTBI LBM 2.0 1–3 340.3 s, autonomy 98.2% Previous-generation LBM 1–6 78.5 s Gen 1 LBM 1–6 22.6 s - LBM 2.0 starts at about 200 s in week 1 and rises to about 400 s by week 3, read off the log axis.
- "Is 340.3 s MTBI good?" The slide's answer: it "could support 50 robots with 3 remote assistants."
- (My reading: the 1k / 10k / 100k-hour models are probably the gen 1, previous-gen, and LBM 2.0 curves, but the slide doesn't say so.)
- Scaling beyond HG-DAgger.
- Question: How do you scale from 1:1 to many operations? (My reading: from one human operator per robot to one operator overseeing many robots.)
- Runtime monitoring is one of the things they're working on. (My guess: it flags when a robot needs a human, so operators don't have to watch every robot.)
- Numbers (uptime, success rates, fleet size, data hours): 33,000 camshafts; LBM 2.0 MTBI 340.3 s at 98.2% autonomy; 50 robots per 3 remote assistants; 4.7B parameters; up to 100k hours of pretraining data; 10 h of demos per task.
Meta Robotics Studio¶
Speaker: Jonathan Thompson, Modelling Lead, Reality Labs. Session and time not recorded.
- Goal: A humanoid robot for the home that is useful and delightful.
- Status: The product isn't ready to share yet, so no robot details, specs, or results.
- The problem at a high level: These are the areas it spans:
- Controls and systems
- Product
- Modelling
- Electrical engineering
- Mechanical engineering
- Safety
- Meta's strengths:
- SLAM and hand tracking
- Egocentric capture
- Vision models
- Muse (see "Plan and act with Muse" below)
- Simulation
- Rapid prototyping
- What Meta is building: Generalized, dexterous robot foundation models. Three parts:
- Learn from people.
- Practice in simulation.
- Plan and act with Muse. Muse plans the task, and a Muse-based policy turns the plan into motion. So it's a two-level split: Muse as the high-level planner and a policy built on it as the low-level controller.
- A data pyramid built on egocentric video, from top to bottom:
- Robot teleop
- UMI grippers
- Egocentric human video
- Aria unlocks egocentric data, for Meta and for its partners.
- FAIR's SAM models ground objects in 3D. They can find and track any concept.
- SuperDex (open source, Meta) simulates contact-rich dexterity.
- Fine-grained manipulation.
- A sponge as a soft actor: deformable objects, not just rigid ones.
AGIBOT: Scaling Robot Intelligence, Building a Physical AI Ecosystem¶
Speaker, session, and time not recorded.
- Framing: "AGIBOT: more than manufacturing." AGIBOT builds a Physical AI ecosystem grounded in real-world robot deployment.
- The ecosystem has three parts:
- Datasets:
- AGIBOT World 2024: Open robot demonstrations that give everyone a shared foundation for learning manipulation skills.
- AGIBOT World 2026: Real-world interactions that connect task decisions to physical execution.
- Latest release: Physical Reasoning. It covers 9 reasoning capabilities.
- Model research:
- GO-3 (preview): a World Reasoning Action Model. It connects reasoning and future prediction with executable robot actions, in three stages:
- Task reasoning: The instruction and the scene guide the prediction.
- Future state prediction: Visual planning predicts what happens next.
- Action generation:
- Related to Day 1's world action models (DreamZero, π0.7's subgoal images), with an explicit reasoning step added in front.
- Genie Evolver: learning through interaction. AGIBOT's claim: "to our knowledge," the first demonstration of collective experience learning, across 96 physical robots doing real-world manipulation tasks.
- Capability benchmarks:
- Robocoliseum: community collaboration. (Spelling to check: the notes have both "Robocoliseum" and "Robotcolliseum".)
- Benchmarks the core capabilities of robot models.
- It brings the community together to build tasks and evaluation protocols that reflect real robot challenges.
Tesla: Building Robot Foundation Models¶
Speaker: Patrick Chao. Session and time not recorded.
Topic: world models for Tesla Optimus. The model is a learned world simulator, used to evaluate and train policies.
- Motivation:
- Evaluating counterfactuals.
- Evaluating new policies in closed loop.
- RL training in closed loop.
- Characteristics the model needs:
- Causal: autoregressive, frame-by-frame prediction.
- Interactive: conditioned on fine-grained actions.
- Consistent: across cameras and across time.
- Low latency: fast enough for closed-loop rollouts during training.
- Physically accurate, as opposed to visually appealing.
- Two kinds of physics to learn, and the data for each:
- Physics of the world: learned from general real-world data.
- Physics of the bot: bootstrap with human data, then fine-tune with Optimus data.
- Scale: Optimus manufacturing is scaling exponentially. (Presumably so robot data grows with the fleet.)
Franka Robotics: Franka Research 3 and FCI¶
Speaker and session not recorded. Slide photos taken at 1:58 and 2:02 pm, right after the Boston Dynamics talk.
- Franka Research 3: The spec icons were too small to read in the photo (payload, reach, workspace coverage, degrees of freedom, integrated torque sensors).
- Programming through FCI (Franka Control Interface), from slide photo:
- Run your own control algorithms on an external real-time PC at 1 kHz, or use the Franka AI Companion.
- libfranka, an open-source C++ library, sends real-time control values at 1 kHz and returns 1 kHz measurements. The diagram also shows Python.
- Integrations: ROS 2, MoveIt, and MATLAB & Simulink.
- Command interfaces: Motion generators take joint position, joint velocity, Cartesian pose, or Cartesian velocity. Controllers take joint torque, joint impedance, or Cartesian impedance.
- The link from the workstation PC to the robot is Ethernet (UDP) at 1 kHz.