Skip to content

Day 2: Tue, Nov 10, 2026

Keynotes

  • Russ Tedrake (MIT; CEO, Walden Robotics), 3:30 pm CT: Learning in the Factories. Title and time come from the @corl_conf keynote post. TechTimes says it covers factory-trained large behavior models, but that is TechTimes' description, not the conference's. The official keynotes page lists no times, so check the on-site schedule.

Sessions attended

  • Boston Dynamics: Single Action Model. Session and speaker not yet recorded. The "Final thoughts" slide photo was taken at 1:56 pm. I found no public paper, blog post, or program listing for it as of Nov 10. Boston Dynamics is a diamond-level CoRL 2026 sponsor, so this may have been a sponsor or industry talk.

  • Franka Robotics: Franka Research 3 and FCI. Speaker and session not yet recorded. Slide photos taken at 1:58 and 2:02 pm. Notes under Live notes.

  • Meta Robotics Studio (Jonathan Thompson, Reality Labs): home humanoid. Session and time not yet recorded. Notes under Live notes.
  • AGIBOT: Scaling Robot Intelligence, Building a Physical AI Ecosystem. Speaker, session, and time not yet recorded. Notes under Live notes.
  • Tesla (Patrick Chao): Building Robot Foundation Models. Session and time not yet recorded. Notes under Live notes.

Papers covered

Paper Group Headline Status Links Weights
DAVATaR UT Austin Fluidic tactile and immersion-depth sensing plus glare and zoom augmentation; 71% vs 41% mean success without tactile summarized page ?
TAM Ai2, KAIST Sim-only residual torque layer below any policy; real Panda box pushing 47.6% to 76.2%, BC flip 50% to 72% summarized arXiv · page · code ✅

Day 2 themes

Cross-day versions: deployment, human data, world models.

  1. Deployment is the product: Tedrake's 33,000 camshafts and 340.3 s between interventions, and Boston Dynamics' "data quality before scale." Lab results aren't enough.
  2. Data pyramids: Meta (teleop, then UMI, then egocentric video) and Tedrake (deployment data will dwarf everything else) both rank data sources. They disagree on what sits on top in the long run.
  3. World models as simulators: Tesla wants a causal, interactive, physically accurate world simulator for evaluation and RL. AGIBOT's GO-3 puts reasoning and future prediction in front of the action.
  4. Low-level layers trained in sim: TAM's residual torque layer works under any policy, and DAVATaR adds tactile sensing for hard perception conditions.

Live notes

Boston Dynamics: Single Action Model

  • Final thoughts: data quality before scale (from slide photo, 1:56 pm):
  • "For whole-body manipulation and for very dexterous manipulation, RL is hard to beat."
  • The photos show Atlas at a workcell and Atlas's hands, in a lab setting.
  • Follow-up Boston Dynamics talks on Thursday, Nov 12 (workshops), from the same slide:
  • Yeuhi Abe and Robin Deits: Our Journey from MPC to RL. 9:50 am, Everything Beneath the Policy workshop.
  • Michael Lutter: Sim-2-Real RL for Manipulation. 2:55 pm, Sim 2 Real 2 Field workshop.

  • Diagrams: Lutter's Day 4 talk expands on both points, with diagrams for the fleet loop and demo resets.

Submerged manipulation: DAVATaR (UT Austin)

Speaker and session not recorded. The paper is from UT Austin (Chin and Martín-Martín labs), not Boston Dynamics.

  • Problem: Submerged tasks are hard. The air–water interface distorts what the camera sees:
  • Refraction: Objects appear shifted and at the wrong scale.
  • Reflection: Glare from the surface hides the scene.
  • Foam: Bubbles and foam on the surface block the view.
  • Paper: DAVATaR: Depth-Aware Visually Augmented Tactile Learning Across the Air-Water Interface. These notes match the paper's Fig. 1. Full summary in papers/davatar.md.

Torque adaptation: TAM (Ai2, KAIST)

Speaker and session not recorded. The project page lists it as a Spotlight.

  • TAM is trained entirely in simulation. It's a residual-torque layer under the low-level controller, so the same weights work under RL, BC, and MPC policies. Full summary in papers/tam.md.

Keynote: Russ Tedrake, Learning from the Factories

The slide title reads "Learning from the factories". The @corl_conf post says "Learning in the Factories".

  • Framing: From the research lab, to deployment, to real customer value.
  • The next hurdle: Turning general-purpose robots into a product that returns real value to customers.
  • Walden Robotics 101:
  • Founded in January 2026 (from slide photo).
  • Walden's approach:
    1. Build the company to deploy and iterate very fast.
    2. Work from a set of core beliefs. Beliefs held when Walden started (from later in the talk):
    3. Base models will get better fast. Plan for it.
    4. Mid-training data is being commoditized, and stronger base models need less of it.
    5. Deployment data is still very valuable. Even a PhD-level manipulator still needs job training in the factory. Stronger base models actually make deployment data more valuable, not less.
    6. The technology is ready. What's left is to scale it and keep advancing the AI. He has high conviction that it will work.
  • The most valuable data is in the field.
  • "It's very reasonable to be skeptical" (from slide photo): "In the limit, deployed robot data will dwarf everything else."

    • Today's pyramid, from top to bottom: teleop; then ego videos, sim, UMI, gloves, and general robot data; then language and vision at the base.
    • Deployment flips it: an inverted funnel with deployment data at the top. Below it sit ego videos, sim, UMI, gloves, and general robot data, then multimodal video, world modelling, language, and reasoning.

    Data pyramid today vs the deployment funnel: deployed robot data on top at scale

  • Manufacturing first, then everything else.

  • Learning from manufacturing experts: Many of the problems robotics needs to solve have been solved before, with different technology and under different constraints.
  • Deployment story: 33,000 camshafts and counting. This is Walden's first and longest-running deployment.
  • The customer sets the requirements.
  • Deploying is more expensive. The extra costs:
  • Field ops and logistics.
  • Contracts, relationships, and trust. "You have to earn it."
  • With a fixed budget for internal robot data: Training distribution ≈ test distribution. My reading, not his words: data collected in deployment matches the test distribution by construction, which is the case for paying the extra cost.
  • Core development vs the deployment flywheel. The tension:
  • Deploying needs 97–99% success.
  • So the task has to be constrained.
  • So the task saturates.
  • So there's no novelty left to carry into the next task.
  • Initial HG-DAgger scaling laws. HG-DAgger is human-gated DAgger: an operator takes over when the policy goes wrong, and those interventions become training data.
  • Question: How does stronger pretraining help in deployment?
  • Finding: Stronger pretrained models need fewer interventions, yet still improve more from them.
  • The interventions are also qualitatively different.
  • Experiment setup (from slide photo):
    • Data mixture: robot demonstrations, egocentric human video, and UMI.
    • Pretraining scales: 1k, 10k, and 100k hours of external data, all on the same 4.7B architecture. These are the generalist models.
    • Task demonstrations: 10 hours per task, which turn each generalist into a task-adapted model.
    • Deployment: Intervention demonstrations collected during deployment feed later model updates.
  • Plot: mean time between interventions (MTBI), log scale, over 6 deployment weeks (from slide photos; values at the last point shown):

    Model Weeks shown MTBI
    LBM 2.0 1–3 340.3 s, autonomy 98.2%
    Previous-generation LBM 1–6 78.5 s
    Gen 1 LBM 1–6 22.6 s
    • LBM 2.0 starts at about 200 s in week 1 and rises to about 400 s by week 3, read off the log axis.
    • "Is 340.3 s MTBI good?" The slide's answer: it "could support 50 robots with 3 remote assistants."
    • (My reading: the 1k / 10k / 100k-hour models are probably the gen 1, previous-gen, and LBM 2.0 curves, but the slide doesn't say so.)
    • Scaling beyond HG-DAgger.
    • Question: How do you scale from 1:1 to many operations? (My reading: from one human operator per robot to one operator overseeing many robots.)
    • Runtime monitoring is one of the things they're working on. (My guess: it flags when a robot needs a human, so operators don't have to watch every robot.)
    • Numbers (uptime, success rates, fleet size, data hours): 33,000 camshafts; LBM 2.0 MTBI 340.3 s at 98.2% autonomy; 50 robots per 3 remote assistants; 4.7B parameters; up to 100k hours of pretraining data; 10 h of demos per task.

Meta Robotics Studio

Speaker: Jonathan Thompson, Modelling Lead, Reality Labs. Session and time not recorded.

  • Goal: A humanoid robot for the home that is useful and delightful.
  • Status: The product isn't ready to share yet, so no robot details, specs, or results.
  • The problem at a high level: These are the areas it spans:
  • Controls and systems
  • Product
  • Modelling
  • Electrical engineering
  • Mechanical engineering
  • Safety
  • Meta's strengths:
  • SLAM and hand tracking
  • Egocentric capture
  • Vision models
  • Muse (see "Plan and act with Muse" below)
  • Simulation
  • Rapid prototyping
  • What Meta is building: Generalized, dexterous robot foundation models. Three parts:
  • Learn from people.
  • Practice in simulation.
  • Plan and act with Muse. Muse plans the task, and a Muse-based policy turns the plan into motion. So it's a two-level split: Muse as the high-level planner and a policy built on it as the low-level controller.
  • A data pyramid built on egocentric video, from top to bottom:
  • Robot teleop
  • UMI grippers
  • Egocentric human video

Meta's data pyramid: robot teleop on top, UMI grippers in the middle, egocentric human video at the base

  • Aria unlocks egocentric data, for Meta and for its partners.
  • FAIR's SAM models ground objects in 3D. They can find and track any concept.
  • SuperDex (open source, Meta) simulates contact-rich dexterity.
  • Fine-grained manipulation.
  • A sponge as a soft actor: deformable objects, not just rigid ones.

AGIBOT: Scaling Robot Intelligence, Building a Physical AI Ecosystem

Speaker, session, and time not recorded.

  • Framing: "AGIBOT: more than manufacturing." AGIBOT builds a Physical AI ecosystem grounded in real-world robot deployment.
  • The ecosystem has three parts:
  • Datasets:
    • AGIBOT World 2024: Open robot demonstrations that give everyone a shared foundation for learning manipulation skills.
    • AGIBOT World 2026: Real-world interactions that connect task decisions to physical execution.
    • Latest release: Physical Reasoning. It covers 9 reasoning capabilities.
  • Model research:
    • GO-3 (preview): a World Reasoning Action Model. It connects reasoning and future prediction with executable robot actions, in three stages:
    • Task reasoning: The instruction and the scene guide the prediction.
    • Future state prediction: Visual planning predicts what happens next.
    • Action generation:
    • Related to Day 1's world action models (DreamZero, π0.7's subgoal images), with an explicit reasoning step added in front.
    • Genie Evolver: learning through interaction. AGIBOT's claim: "to our knowledge," the first demonstration of collective experience learning, across 96 physical robots doing real-world manipulation tasks.
  • Capability benchmarks:
    • Robocoliseum: community collaboration. (Spelling to check: the notes have both "Robocoliseum" and "Robotcolliseum".)
    • Benchmarks the core capabilities of robot models.
    • It brings the community together to build tasks and evaluation protocols that reflect real robot challenges.

Tesla: Building Robot Foundation Models

Speaker: Patrick Chao. Session and time not recorded.

Topic: world models for Tesla Optimus. The model is a learned world simulator, used to evaluate and train policies.

  • Motivation:
  • Evaluating counterfactuals.
  • Evaluating new policies in closed loop.
  • RL training in closed loop.
  • Characteristics the model needs:
  • Causal: autoregressive, frame-by-frame prediction.
  • Interactive: conditioned on fine-grained actions.
  • Consistent: across cameras and across time.
  • Low latency: fast enough for closed-loop rollouts during training.
  • Physically accurate, as opposed to visually appealing.
  • Two kinds of physics to learn, and the data for each:
  • Physics of the world: learned from general real-world data.
  • Physics of the bot: bootstrap with human data, then fine-tune with Optimus data.
  • Scale: Optimus manufacturing is scaling exponentially. (Presumably so robot data grows with the fleet.)

Franka Robotics: Franka Research 3 and FCI

Speaker and session not recorded. Slide photos taken at 1:58 and 2:02 pm, right after the Boston Dynamics talk.

  • Franka Research 3: The spec icons were too small to read in the photo (payload, reach, workspace coverage, degrees of freedom, integrated torque sensors).
  • Programming through FCI (Franka Control Interface), from slide photo:
  • Run your own control algorithms on an external real-time PC at 1 kHz, or use the Franka AI Companion.
  • libfranka, an open-source C++ library, sends real-time control values at 1 kHz and returns 1 kHz measurements. The diagram also shows Python.
  • Integrations: ROS 2, MoveIt, and MATLAB & Simulink.
  • Command interfaces: Motion generators take joint position, joint velocity, Cartesian pose, or Cartesian velocity. Controllers take joint torque, joint impedance, or Cartesian impedance.
  • The link from the workstation PC to the robot is Ethernet (UDP) at 1 kHz.