双手机器人工具使用失败恢复诊断：哪只手迟了？
Which Hand Was Late? Diagnosis for Robot Failure Recovery in Bimanual Tool Use
AI 导读
arXiv 一项研究针对双手机器人工具使用任务中的失败诊断问题，指出工具打滑、双手手势时序错位、多重原因并发等失败类型与单臂场景不同。实验测量各感知通道能区分哪些失败原因：工具打滑可从工具手读出，哪只手迟了需读双臂，迟手过弱推力在被动感知下不可识别。VLM 内部状态几乎完美携带迟手信息，但答案是否报告取决于提问措辞。
正文 · 原文
Abstract:Failure diagnosers for robot manipulation are increasingly vision--language models (VLMs), trained and evaluated on single-arm failures produced by perturbing a successful demonstration. Bimanual tool use adds failures of other kinds: a tool that slips out of its socket while still held, two correct gestures made at different times, and attempts that fail for two reasons at once. On a simulated bimanual tool-use task whose failure causes are read from the recorded trace, we measure what each sensing channel lets a diagnoser tell. The tool's slip is read on the tool hand; which hand was late is read on both arms, while a sensor on one hand reads its own hand's delay and at most part of the other's; and whether a late attempt was also pushed too weakly is not identifiable from passive sensing when the tool hand is late. Within two attempts, a repair policy that escalates when the cause is not identifiable repairs as many failures as one told the true cause. A VLM's internal state carries the late hand almost perfectly, along a direction no closer to the one its answer reads than a random one; whether the answer reports it depends on the wording of the question.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37533 [cs.RO] |
| (or arXiv:2610.37533v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37533 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Baptiste Bonin [view email]
[v1]
Tue, 27 Oct 2026 16:53:18 UTC (2,605 KB)
来源：arXiv cs.AI · arxiv.org