arXiv cs.AI· Shunming Liu, Jianyi Peng, Cheng Li, Lu Hou, Pei Huang, Yajing Sun, Huaxiao Liu·· 1 小时前AI 评分32
MLLM 能否理解环视驾驶视频？多视角自动驾驶 QA 基准 MVDriveBench
Can MLLMs Reason over Surround-View Driving Videos? A Multi-View Autonomous Driving QA Benchmark
AI 导读
研究者提出 MVDriveBench，一个面向自动驾驶环视视频理解的多视角视频 QA 基准，包含 12 类任务，覆盖从底层视觉感知到高层驾驶决策的完整推理链。该基准评估了 23 个开源、闭源及专家 MLLM，结果显示现有模型环视时空推理能力仍不足。在 AlpaSim 中的闭环评估表明，环视时空推理能力与闭环驾驶表现存在相关性。
正文 · 原文
Abstract:Surround-view video understanding in autonomous driving is a crucial capability for Multimodal Large Language Models (MLLMs) to achieve reliable driving reasoning. Real-world driving often requires information beyond the front-view camera. Existing MLLMs primarily focus on front-view information, while their spatio-temporal understanding and reasoning abilities remain unknown. To address this gap, we propose MVDriveBench, a high-information-density video QA benchmark for autonomous driving, characterized by synchronized surround-view inputs, spatio-temporal understanding, and reasoning-chain-oriented evaluation. We design 12 categories of QA tasks that cover the complete reasoning pipeline from low-level visual perception to high-level driving decision-making, enabling a systematic evaluation of MLLMs surround-view spatio-temporal understanding and reasoning capabilities in complex driving environments. We conduct a comprehensive evaluation of 23 open-source, closed-source, and expert MLLMs on MVDriveBench. The results show that current MLLMs still exhibit insufficient surround-view spatio-temporal reasoning ability. Meanwhile, closed-loop evaluations of specialist models in AlpaSim reveal a correlation between surround-view spatio-temporal reasoning capabilities and closed-loop driving performance, highlighting the importance of comprehensive multi-view understanding for reliable vehicle control.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15225 [cs.AI] |
| (or arXiv:2611.15225v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15225 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shunming Liu [view email]
[v1]
Wed, 11 Nov 2026 09:18:15 UTC (3,566 KB)
来源：arXiv cs.AI · arxiv.org