arXiv cs.AI· Madelon Bernardy, Mohammad Mahdi Derakhshani, David M Knigge, Riccardo Valperga, Artem Moskalev·· 4 小时前AI 评分35
形状教师对齐方法让图像策略机器人获得 3D 感知并加速收敛
Aligning Policy Encoders with a Shape Teacher
AI 导读
一项 arXiv 研究提出 shape-teacher alignment，在训练时用 Point-MAE 预训练 3D 特征作为对齐目标，让图像策略编码器习得几何结构，部署时仍只需图像输入、无需 3D 传感或深度输入。在 LIBERO 基准上，预训练策略在 Object 和 Spatial 套件中的收敛分别加速 4.7x 和 5.6x，成功率提升，图像表征更多围绕物体形状及对应部位组织。
正文 · 原文
Abstract:3D-aware representations can improve robotic manipulation by providing geometric information that is robust to changes in viewpoint, object position, and occlusion. However, using 3D inputs at deployment requires additional sensing and processing, making image-based policies more practical. We ask whether an image-based policy can acquire 3D shape information during training while retaining a purely image-based interface at deployment. We propose shape-teacher alignment, a training-time mechanism that uses a pretrained 3D shape representation as an alignment target for an image-based policy encoder. Specifically, we use Point-MAE features from individual objects as targets for corresponding image patches, transferring geometric structure to the policy without changing its deployment inputs or architecture. On LIBERO, shape-teacher alignment produces image representations that are more structured around object shape and corresponding parts, while substantially accelerating policy convergence and improving success rates. For pretrained policies, convergence is accelerated by 4.7x and 5.6x on the Object and Spatial suites, respectively, without requiring the shape teacher or depth input at deployment.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2611.08275 [cs.RO] |
| (or arXiv:2611.08275v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08275 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Angela Bernardy [view email]
[v1]
Thu, 5 Nov 2026 16:44:24 UTC (2,329 KB)
来源：arXiv cs.AI · arxiv.org