TouchSafeBench 基准测试：检测视觉语言模型在人机协作中的碰撞理解能力
Probing Collision Grounding in Vision-Language Models for Safe Human-Robot Collaboration
AI 导读
研究者提出"碰撞理解"（collision grounding）概念，并发布基于 Habitat 3.0 构建的 TouchSafeBench 基准，包含 2,940 段模拟室内共处场景，覆盖社交导航与社交重排两类任务。
正文 · 原文
Abstract:Safe human--robot collaboration requires more than visual description: a monitor must determine whether the robot body is safely separated, already colliding with the scene or a person, or about to collide. We call this capability collision grounding: binding visual observations to robot body geometry, camera viewpoint, scene layout, human proximity, and temporal motion in order to infer present and imminent contact. We introduce TouchSafeBench, a physics-grounded benchmark for evaluating collision grounding in vision-language models (VLMs). Built in Habitat~3.0, TouchSafeBench contains 2,940 simulated indoor co-presence episodes across social navigation and social rearrangement, with synchronized multi-view RGB-D observations, top-down trajectory maps, calibrated camera metadata, and simulator-derived contact labels. We study two deployment-facing tasks: classifying the current safety state and warning about imminent collision before contact. Across three frontier or robotics-oriented VLMs and nine visual representations, current models remain far from reliable: the best average Macro-F1 stays below 50\%, explicit depth is not automatically transformed into robot-body collision evidence, and robot--scene contact is consistently harder than human-contact risk. TouchSafeBench reveals a central limitation of embodied VLMs: visual fluency does not imply physical accountability. Reliable robot safety monitors will need representations that explicitly bind viewpoint, robot morphology, metric geometry, and future collision. The benchmark is available at this https URL .
| Comments: | Accepted by 10th Conference on Robot Learning (CoRL 2026), Austin TX, USA |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Robotics (cs.RO) |
| Cite as: | arXiv:2605.31196 [cs.CV] |
| (or arXiv:2605.31196v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.31196 arXiv-issued DOI via DataCite |
Submission history
From: Xiaohao Xu [view email]
[v1]
Fri, 29 May 2026 12:04:38 UTC (7,088 KB)
[v2]
Fri, 23 Oct 2026 10:43:59 UTC (6,034 KB)
来源：arXiv cs.AI · arxiv.org