Skip to content

RoboTTT: Context Scaling for Robot Policies

Session: Oral, Day 1. Reported as the Best Student Paper Award winner, announced Nov 11 (Day 3); corl.org's awards page didn't list winners when checked. Speaker(s): Paper: arXiv 2607.15275 (HTML, PDF) by Yunfan Jiang et al. (NVIDIA GEAR, with Stanford and UT Austin), July 2026 Project page: research.nvidia.com/labs/gear/robottt

Paper summary (from the arXiv HTML, abstract, and project page; the PDF was too large to fetch)

Authors: Yunfan Jiang, Yevgen Chebotar, Ruijie Zheng, Fengyuan Hu, Yunhao Ge, Jimmy Wu, Tianyuan Dai, Scott Reed, Li Fei-Fei, Yuke Zhu, and Linxi "Jim" Fan (NVIDIA, Stanford, UT Austin). Fei-Fei, Zhu, and Fan are equal advisors. Submitted July 16, 2026.

  • Core idea: Test-Time Training (TTT) extends a robot policy's visuomotor context to 8K timesteps, about 5 minutes at 30 Hz. Fast weights carry the history forward over time, so the cost of each inference step stays constant. A Transformer's cost grows with history length, even with a KV cache.

Architecture

  • Base model: GR00T N1.7, a VLM backbone with a Diffusion Transformer (DiT) flow-matching action head.
  • Where TTT goes: One TTT layer is added to each of the 16 DiT layers, after self- and cross-attention. Attention handles the tokens within one timestep, and TTT handles information across timesteps.
  • Fast model: Each fast model is a two-layer MLP. It is updated by a gradient step on a key–value binding loss (TTT-KVB), with a learnable inner learning rate η.
  • Gating: O = tanh(α) ⊙ O_TTT + O_attn, with α initialized at 0.001, so training starts from the base VLA's behavior.
  • Tokens for each timestep: 16 learned register tokens, a proprioception token, and the noised action tokens. Vision–language tokens are only cross-attended and do not pass through TTT, which keeps cost down. The register tokens carry vision–language information across time.
  • Observations: 4 RGB cameras (top, bottom, and both wrists), proprioception, and language.
  • Parameters (unverified): A secondary summary says 538M for the DiT and 690M total. Check this against the PDF.

Training

  • Pretraining: Bimanual tabletop robot data plus egocentric human data. Context grows gradually from 128 timesteps up to 8K, over 30K steps on 16 GB200 GPUs.
  • Post-training: 20K steps at 1K context for each task. The real-robot data is 8 h for Pup Go Car, 6 h for Circuit, and 5 h for Gear Bot, plus 30 min of perturbation data. Inference runs at 8K context.
  • Sequence action forcing: The noise level is sampled independently for each action chunk: τ = s(1−u), with u ~ Beta(1.5, 1) and s = 0.999. The authors report that one shared noise level across the sequence makes training unstable.
  • Truncated BPTT: Sequences are split into fixed-length segments. Gradients stop at segment boundaries, but the fast weights carry across them. The segment length is not in the HTML text.

Experiments (YAM bimanual robot; 20 trials per task, or 10 for Gear Bot)

  • Tasks:
  • Pup Go Car: car assembly, episodes of about 2 min.
  • Circuit: 80 configurations, 20 used for training and 60 held out for testing, episodes of about 1 min.
  • Gear Bot: 10-stage assembly, episodes of about 5 min.
  • Baselines:
  • GR00T N1.7 with a single-step context.
  • GR00T N1.7 Hist., which adds 1 history frame.
  • GDN, a Gated DeltaNet in place of the TTT layers.
Task completion (%) RoboTTT GR00T N1.7 N1.7 Hist. GDN
Pup Go Car 89 57 40 57
Circuit 78 23 54 53
Gear Bot 71 47 54 59
Average 79 42 49 56
Full successes RoboTTT GR00T N1.7 N1.7 Hist. GDN
Pup Go Car (/20) 9 3 0 3
Circuit (/20) 13 3 8 8
Gear Bot (/10) 2 0 0 0
  • Headline results:
  • RoboTTT averages 79%, which is 87% higher than the single-step baseline (42%) and 41% higher than the best baseline, GDN (56%).
  • It is the only method with any full success on Gear Bot.
  • Context scaling: Pretraining at 8K context reaches 71.5%, compared with 43.9% for the same model pretrained at 1K. That is +62% in the abstract and +63% in the body. The best short-context baseline reaches 45.6%. GDN shows no improvement as context grows. The authors say this is the first time closed-loop performance has been shown to keep improving with pretraining context length.
  • One-shot imitation from a human video (Circuit, unseen configuration): RoboTTT scores 65% and succeeds fully in 6/10 trials, against 33% and 0/10 for GDN. GDN is the only baseline here, because GR00T N1.7 cannot condition on a video in its context.
  • Perturbation recovery:
  • Roof: 15/20 for RoboTTT, 10/20 for N1.7, 3/20 for Hist., and 13/20 for GDN.
  • Tire: 18/20, 11/20, 5/20, and 18/20 in the same order.
  • The "83% vs 53%" in the intro matches pooling both types for RoboTTT and N1.7. GDN pooled is close, at 77.5%.
  • On-the-fly improvement (DAgger Distillation): RoboTTT is fine-tuned on Pup Go Car with 100 DAgger trajectories. The authors say it is 36% better than the same model without this training.
  • Latency and memory: Constant per-step cost is claimed, but no measured latency or memory comparison with Transformers appears in the HTML text.

Limitations and caveats

  • Trial counts are small: 10–20 per condition.
  • Below 1K context, RoboTTT falls short of its longer-context variants, because 1K (about 30 s) is shorter than every task episode.
  • The authors only hypothesize, without testing it, that the update rule explains the gap over GDN.
  • Training cost still grows with context length (unverified).

Talk notes

Robo TTT in a nutshell

  • RoboTTT treated as sequence modeling

What RoboTTT enables

  1. Learning from long-context demonstrations.
  2. Building robust features on the fly.
  3. Tracking its own progress through a task.

RoboTTT fast-weight recurrence and constant per-step cost