RoboTTT: Context Scaling for Robot Policies¶
Session: Oral, Day 1. Reported as the Best Student Paper Award winner, announced Nov 11 (Day 3); corl.org's awards page didn't list winners when checked. Speaker(s): Paper: arXiv 2607.15275 (HTML, PDF) by Yunfan Jiang et al. (NVIDIA GEAR, with Stanford and UT Austin), July 2026 Project page: research.nvidia.com/labs/gear/robottt
Paper summary (from the arXiv HTML, abstract, and project page; the PDF was too large to fetch)¶
Authors: Yunfan Jiang, Yevgen Chebotar, Ruijie Zheng, Fengyuan Hu, Yunhao Ge, Jimmy Wu, Tianyuan Dai, Scott Reed, Li Fei-Fei, Yuke Zhu, and Linxi "Jim" Fan (NVIDIA, Stanford, UT Austin). Fei-Fei, Zhu, and Fan are equal advisors. Submitted July 16, 2026.
- Core idea: Test-Time Training (TTT) extends a robot policy's visuomotor context to 8K timesteps, about 5 minutes at 30 Hz. Fast weights carry the history forward over time, so the cost of each inference step stays constant. A Transformer's cost grows with history length, even with a KV cache.
Architecture
- Base model: GR00T N1.7, a VLM backbone with a Diffusion Transformer (DiT) flow-matching action head.
- Where TTT goes: One TTT layer is added to each of the 16 DiT layers, after self- and cross-attention. Attention handles the tokens within one timestep, and TTT handles information across timesteps.
- Fast model: Each fast model is a two-layer MLP. It is updated by a gradient step on a key–value binding loss (TTT-KVB), with a learnable inner learning rate η.
- Gating: O = tanh(α) ⊙ O_TTT + O_attn, with α initialized at 0.001, so training starts from the base VLA's behavior.
- Tokens for each timestep: 16 learned register tokens, a proprioception token, and the noised action tokens. Vision–language tokens are only cross-attended and do not pass through TTT, which keeps cost down. The register tokens carry vision–language information across time.
- Observations: 4 RGB cameras (top, bottom, and both wrists), proprioception, and language.
- Parameters (unverified): A secondary summary says 538M for the DiT and 690M total. Check this against the PDF.
Training
- Pretraining: Bimanual tabletop robot data plus egocentric human data. Context grows gradually from 128 timesteps up to 8K, over 30K steps on 16 GB200 GPUs.
- Post-training: 20K steps at 1K context for each task. The real-robot data is 8 h for Pup Go Car, 6 h for Circuit, and 5 h for Gear Bot, plus 30 min of perturbation data. Inference runs at 8K context.
- Sequence action forcing: The noise level is sampled independently for each action chunk: τ = s(1−u), with u ~ Beta(1.5, 1) and s = 0.999. The authors report that one shared noise level across the sequence makes training unstable.
- Truncated BPTT: Sequences are split into fixed-length segments. Gradients stop at segment boundaries, but the fast weights carry across them. The segment length is not in the HTML text.
Experiments (YAM bimanual robot; 20 trials per task, or 10 for Gear Bot)
- Tasks:
- Pup Go Car: car assembly, episodes of about 2 min.
- Circuit: 80 configurations, 20 used for training and 60 held out for testing, episodes of about 1 min.
- Gear Bot: 10-stage assembly, episodes of about 5 min.
- Baselines:
- GR00T N1.7 with a single-step context.
- GR00T N1.7 Hist., which adds 1 history frame.
- GDN, a Gated DeltaNet in place of the TTT layers.
| Task completion (%) | RoboTTT | GR00T N1.7 | N1.7 Hist. | GDN |
|---|---|---|---|---|
| Pup Go Car | 89 | 57 | 40 | 57 |
| Circuit | 78 | 23 | 54 | 53 |
| Gear Bot | 71 | 47 | 54 | 59 |
| Average | 79 | 42 | 49 | 56 |
| Full successes | RoboTTT | GR00T N1.7 | N1.7 Hist. | GDN |
|---|---|---|---|---|
| Pup Go Car (/20) | 9 | 3 | 0 | 3 |
| Circuit (/20) | 13 | 3 | 8 | 8 |
| Gear Bot (/10) | 2 | 0 | 0 | 0 |
- Headline results:
- RoboTTT averages 79%, which is 87% higher than the single-step baseline (42%) and 41% higher than the best baseline, GDN (56%).
- It is the only method with any full success on Gear Bot.
- Context scaling: Pretraining at 8K context reaches 71.5%, compared with 43.9% for the same model pretrained at 1K. That is +62% in the abstract and +63% in the body. The best short-context baseline reaches 45.6%. GDN shows no improvement as context grows. The authors say this is the first time closed-loop performance has been shown to keep improving with pretraining context length.
- One-shot imitation from a human video (Circuit, unseen configuration): RoboTTT scores 65% and succeeds fully in 6/10 trials, against 33% and 0/10 for GDN. GDN is the only baseline here, because GR00T N1.7 cannot condition on a video in its context.
- Perturbation recovery:
- Roof: 15/20 for RoboTTT, 10/20 for N1.7, 3/20 for Hist., and 13/20 for GDN.
- Tire: 18/20, 11/20, 5/20, and 18/20 in the same order.
- The "83% vs 53%" in the intro matches pooling both types for RoboTTT and N1.7. GDN pooled is close, at 77.5%.
- On-the-fly improvement (DAgger Distillation): RoboTTT is fine-tuned on Pup Go Car with 100 DAgger trajectories. The authors say it is 36% better than the same model without this training.
- Latency and memory: Constant per-step cost is claimed, but no measured latency or memory comparison with Transformers appears in the HTML text.
Limitations and caveats
- Trial counts are small: 10–20 per condition.
- Below 1K context, RoboTTT falls short of its longer-context variants, because 1K (about 30 s) is shorter than every task episode.
- The authors only hypothesize, without testing it, that the update rule explains the gap over GDN.
- Training cost still grows with context length (unverified).
Talk notes¶
Robo TTT in a nutshell
- RoboTTT treated as sequence modeling
What RoboTTT enables
- Learning from long-context demonstrations.
- Building robust features on the fly.
- Tracking its own progress through a task.