VLM 视觉定位评估：面向视障人群的 VizWiz-VQA-Grounding 基准研究
Evaluating Visual Language Models on Visual Grounding for Visually Impaired People
AI 导读
一项研究评估开源 VLM 在视障人群辅助场景下的视觉定位能力，使用 VizWiz-VQA-Grounding 数据集与挑战赛数据，并提出 VG-SHAP 方法，将现有可解释性框架扩展用于黑盒 VLM 的视觉定位解释。结果显示，VLM 虽能生成正确答案，却难以定位到图像中的正确区域；该结论受可解释性框架中分割模型质量的限制，仍需进一步研究。
正文 · 原文
Abstract:Visual grounding (VG) in visual question answering (VQA) is a fundamental capability of Vision-Language Models (VLMs), enabling the detection of the most relevant region in an image to answer the question. This reflects the model's capability of multimodal comprehension, which is crucial when interacting with visually impaired people (VIP). Indeed, visual grounding, when applied to assistive scenarios, is not merely a technical feature, but rather a trust and safety requirement. In this work, we study visual grounding in VLMs for visually impaired people. To this end, we assess open-source VLMs on visual grounding using the VizWiz-VQA-Grounding dataset and challenge, which contains real-life images taken by VIP. We propose VG-SHAP, which extends an existing explainability framework to provide visual grounding explanations, allowing the evaluation of black-box VLMs. Our results on the VizWiz-VQA-Grounding challenge show that despite generating correct answers, VLMs struggle to ground in the proper region in the image. However, these results are constrained by the quality of the segmentation model of our explainability framework and further research is necessary.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33078 [cs.CV] |
| (or arXiv:2610.33078v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33078 arXiv-issued DOI via DataCite |
Submission history
From: Lydia Achour [view email]
[v1]
Wed, 21 Oct 2026 08:56:07 UTC (1,045 KB)
来源：arXiv cs.AI · arxiv.org