WAM-TTT：通过观看人类游玩视频在测试时引导世界-动作模型
WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time
研究者提出 WAM-TTT，一种从原始人类视频对世界-动作模型（WAM）进行测试时训练的引导框架。该方法通过自监督视频预测将人类视频吸收进冻结 WAM 内的轻量自适应记忆，并借助配对人-机数据的元训练和键值记忆重建目标使记忆可用于控制。测试时仅需无标注人类视频即可适配，无需机器人动作、人工标注或任务特定微调，在多样操作任务上持续优于上下文人类视频条件化基线。
Authors:Yusen Feng, Bingchen Han, Jiangran Lyu, Kai Liu, Yixin Zheng, Yuxuan Wan, Weiheng Liu, Sun Han, Ruiqin Li, Yulong Zhang, Fangfu Liu, Xuesong Shi, Libin Liu, Yizhou Wang, Zhizheng Zhang, He Wang
Abstract:Steering robot foundation models (RFMs) toward new task variants or user-preferred behaviors remains challenging, often requiring additional robot demonstrations, task-specific fine-tuning, or long-context conditioning. We present WAM-TTT, a test-time training framework for steering world action models from raw human videos. Rather than treating human videos as trajectories to imitate, WAM-TTT absorbs them into a lightweight adaptive memory inside a frozen WAM through self-supervised video prediction. To make this memory useful for control, we introduce a meta-training stage that aligns human demonstrations with robot behaviors using paired human-robot data and a key--value memory reconstruction objective. At test time, only unlabeled human videos are required to adapt the memory, while the pretrained WAM remains frozen. This enables efficient and reusable steering without robot actions, human-side annotations, or task-specific fine-tuning, while preserving the generalization ability of the foundation model. Extensive experiments show that WAM-TTT consistently outperforms in-context human-video conditioning baselines across diverse manipulation tasks and generalization settings.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.06988 [cs.RO] |
| (or arXiv:2607.06988v3 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2607.06988 arXiv-issued DOI via DataCite |
Submission history
From: Kai Liu [view email]
[v1]
Wed, 8 Jul 2026 04:23:22 UTC (10,111 KB)
[v2]
Fri, 10 Jul 2026 02:22:52 UTC (10,112 KB)
[v3]
Fri, 23 Oct 2026 06:57:20 UTC (10,585 KB)
来源：arXiv cs.AI · arxiv.org