TAM: Torque Adaptation Module for Robust Motion Transfer in Manipulation¶
Session: CoRL 2026 Spotlight (from the project page; the paper text doesn't state it). Day, time, and room not recorded. Speaker(s): Paper: arXiv 2606.06218 (v3, Oct 2026) by Dongwon Son (KAIST; work done during an internship at Ai2), Florian Shkurti (Ai2, Toronto), Jason Lee (Ai2, UW), Naman Shah (Ai2), Beomjoon Kim (KAIST), and Dieter Fox (Ai2, UW). Kim and Fox jointly supervised. Project page: dongwon-son.github.io/tam-project-page · Code: Dongwon-Son/TAM (MuJoCo/MJX + JAX). According to the README, a multi-robot inference checkpoint is released, but the robot-side controller bridge is not. License not checked.
Paper summary (from the arXiv PDF text; figures not inspected)¶
- Problem: A policy tuned for one robot behaves differently on another because of the sim-to-real gap, unknown payloads, or instance-to-instance differences. In dynamic, contact-rich tasks small motion errors break contact timing.
- Domain randomization makes policies conservative, and system identification needs data recollected for each robot or payload.
Method
- Where it sits: Between the low-level controller and the robot's torque interface. The controller turns the policy's action (joint targets, end-effector targets, or torques) into a nominal torque τ0. TAM adds a residual Δτ, and the result is clipped to joint torque limits.
- The policy is untouched. TAM never sees policy observations, rewards, or the action space, so the same weights sit under RL, BC, and MPC policies.
- Two parts:
- History encoder: A causal transformer over a 12 s, 1 kHz proprioceptive history (q, q̇, applied torque, plus a physics-residual feature eτ from the ideal inverse model). It uses patch tokenization (400/200 samples) and runs at about 5 Hz on a workstation, producing a latent z.
- Torque adaptor: Takes a short recent window, τ0, and z. It runs at 1 kHz on the controller CPU.
- Before the first latent is ready (about 0.5 s), the nominal torque passes through unchanged.
- Trained entirely in randomized simulation, with no real-robot data:
- Task-agnostic free-space joint-motion rollouts with random waypoints and injected smooth external torques. No task demonstrations.
- The target is the torque an ideal robot needs to reproduce the motion, with a Huber loss.
- Randomization (Panda): body mass ×U[0.9, 1.1], payload U[0, 1.5] kg, viscous damping, sigmoidal friction up to 3 Nm, torque bias and dead zone, plus history noise and 0–4 ms delays.
- Pipeline: Pretraining on 5 robots (Panda, Piper, iiwa14, Flexiv Rizon4, RBY-1), then optional robot-specific sim fine-tuning. A DAgger variant fine-tunes on TAM-corrected rollouts.
Results (zero-shot on a real Franka Panda)
| Task | Ideal sim | Direct | Online SysID | RMA | TAM (w/ RMA) | TAM |
|---|---|---|---|---|---|---|
| RL box pushing (21 trials) | 84.0% | 47.6% | 57.1% | 52.4% | 66.7% | 76.2% |
| BC cube flip, ACT (50 trials) | 73.5% | 50.0% | 34.0% | – | – | 72.0% |
| MPC ball-on-plate, goals per 15 s (20 runs) | 15.9 ± 2.3 | 8.8 ± 2.4 | 13.1 ± 2.3 | – | – | 13.6 ± 2.9 |
- End-effector tracking RMSE (cm, 10 trials):
- Real Panda: Direct 4.09, PhysPred 1.74 / 1.64, TAM 1.51 / 1.04 (reset / carried adaptation state).
- With an unknown 1 kg payload: Direct 6.04, TAM 2.35 / 1.29.
- Cross-robot transfer (sim, joint RMSE °, mean over 7 robots): Direct 4.48, Online SysID 1.24, TAM 0.76, TAM fine-tuned 0.62. TAM also works on robots it never trained on: Google Robot 4.69 → 1.05, Unitree Z1 6.11 → 0.93.
- Ablations (sim box pushing, 100 trials):
- Direct 21%, short-window adaptor only 42%, no eτ 67%, predicting explicit physics parameters 59%, RMA-style oracle-latent supervision 70%, full TAM 84%.
- Predicting the torque directly beats predicting physics parameters.
- Out-of-distribution friction: At 3× the training bound, TAM still gets 80% against 59% for Direct.
- Limitations (as the authors state them):
- Needs a torque interface, an ideal inverse model, and matching kinematics and actuator order.
- Corrections are one-step and local, with no long-horizon guarantee.
- Clipping to joint torque limits caps which payloads it can handle.
- Contacts outside the training distribution can be mistaken for robot-side mismatch.
Talk notes¶
- TAM is trained entirely in simulation.
Takeaways for our work¶
- It sits below the policy, so it could in principle correct dynamics mismatch for any policy we deploy without retraining that policy. It needs torque-level access, though.