跳到正文
arXiv cs.AI· Jason Armitage, Ioannis Tsochantaridis, Linda Mazzone, Chuqiao Yan, Srini Narayanan, Sarah Ebling·· 2 小时前AI 评分34

MAP:首个面向真实世界场所无障碍出行规划的多模态 AI 评测基准

MAP: A Benchmark on Multimodal Accessibility Planning for Real World Places

AI 导读

研究者推出 MAP 基准,首次评测多模态 AI 系统在为有无障碍需求的用户规划真实场所出行时的表现。该基准包含两项新评估:验证场所无障碍信息的声明核验,以及检索视觉证据的能力;通过定时刷新真实数据并采用自动加人工评分,支持对不同 AI 系统的比较。

正文 · 原文

View PDF HTML (experimental)

Abstract:We introduce MAP, the first benchmark to evaluate multimodal AI systems as assistants for users with accessibility requirements when planning visits to places in the real world. In our evaluation, systems are presented with requests to verify or recommend a point of interest meeting an accessibility requirement. MAP contains two novel assessments: Claim verification for accessibility planning assesses if information on places and stated accessibility features is supported and identifies places that satisfy requested accessibility features. Visual evidence retrieval for accessibility planning checks if a multimodal AI system can select visual evidence for the requested place and accessibility feature. Our methodology supports comparison of AI systems in a setting where place information and accessibility information can change over time by evaluating systems and refreshing ground truth data at scheduled times. The benchmark is based on automatic rating and human rating for a proportion of responses.
Subjects: Artificial Intelligence (cs.AI)
ACM classes: I.2.7; I.2.10
Cite as: arXiv:2608.28384 [cs.AI]
  (or arXiv:2608.28384v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2608.28384

arXiv-issued DOI via DataCite

Submission history

From: Jason Armitage [view email]
[v1] Fri, 28 Aug 2026 14:41:24 UTC (9,853 KB)
[v2] Wed, 18 Nov 2026 12:53:35 UTC (7,379 KB)

来源:arXiv cs.AI · arxiv.org