InterTrack 行为世界模型实现人形机器人跨地形全身跟踪与跌倒恢复
GigaBrain-WBC-0.5: A Behavior World Model for Robust Humanoid Whole-Body Tracking with Environment Interaction
研究团队提出 InterTrack，首个支持环境交互的鲁棒人形全身跟踪行为世界模型，Transformer 联合预测下一动作、状态和行为分布，并用自动标注管线从重定向动作重建 3D 支撑几何以扩展训练数据。该方法在复杂地形交互上成功率达 81.3%，为最佳基线的 4.3 倍，跌倒恢复率 99.3%，并首次实现人形机器人实时跨地形全身遥操作。
Authors:Ziyang Cheng, Tianshu Tang, Jinxin Lan, Xinze Chen, Yuhan Gong, Zhichao Liu, Changzhong Wu, Yahao Mao, Zongyan Deng, Mingxuan Ma, Huasen Xi, Yilong Liu, Yutong Wu, Xiaofeng Wang, Borui Zhang, Bingyao Yu, Yang Wang, Yun Ye, Guan Huang, Xiaojie Jin, Zheng Zhu, Jiwen Lu
Abstract:General-purpose motion trackers enable humanoid robots to follow diverse whole-body motions while maintaining balance, but are trained only on flat ground, failing to exploit bipedal mobility over complex terrain. Cross-terrain controllers, meanwhile, are task-specific or accept only low-dimensional locomotion commands. We introduce InterTrack, the first behavior world model (BWM) for robust whole-body tracking with environment interaction. Its Transformer jointly predicts the next action, state, and behavior distribution, learning environment-conditioned dynamics. To scale interaction training data, an automatic annotation pipeline reconstructs 3D support geometry from retargeted motions. At deployment, the policy handles commands implausible in the current environment in a "best-effort" manner. Quantitatively, InterTrack achieves an 81.3% success rate on terrain interaction (4.3 times the best evaluated baseline) and a 99.3% fall-recovery rate, while also improving free-space tracking and outperforming three leading tracking baselines across all of these regimes. To our knowledge, we provide the first demonstration of real-time cross-terrain whole-body teleoperation on a humanoid robot, alongside object interaction, stable responses to missing supports, and robust recovery from falls.
| Comments: | Technical report. Project page: this https URL Code: this https URL Data: this https URL |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.18234 [cs.RO] |
| (or arXiv:2608.18234v5 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2608.18234 arXiv-issued DOI via DataCite |
Submission history
From: Ziyang Cheng [view email]
[v1]
Tue, 18 Aug 2026 18:21:36 UTC (16,562 KB)
[v2]
Sun, 23 Aug 2026 09:07:15 UTC (15,875 KB)
[v3]
Fri, 11 Sep 2026 05:22:00 UTC (15,875 KB)
[v4]
Thu, 17 Sep 2026 07:45:26 UTC (21,107 KB)
[v5]
Thu, 5 Nov 2026 07:54:51 UTC (20,969 KB)
来源：arXiv cs.AI · arxiv.org