面向自动驾驶语义异常的动作条件 VLM 监控研究
Action-Conditioned VLM Monitoring for Autonomous Vehicle Semantic Anomalies
AI 导读
一项 arXiv 研究提出动作条件化的自动驾驶行为监控方法：VLM 先从隐藏结局的视频窗口预测场景所需动作，再与实际自车运动轨迹按显式兼容规则比较，判断执行动作是否恰当。
正文 · 原文
Abstract:We study action-conditioned monitoring of autonomous-vehicle behavior in semantically unusual driving scenes. The target is whether the executed ego action is appropriate for the scene: the same scene can admit both normal and anomalous behavior, so visual unusualness alone does not determine the monitoring label. A Vision-Language Model (VLM) first predicts the scene-required action from an outcome-hidden video window; the monitor then compares that expectation with the realized ego-motion trajectory under an explicit compatibility rule. To study this monitoring task, we contribute a six-field taxonomy of 112 paired scenarios spanning both failure polarities (42 derived from documented incidents), a generated video corpus with estimated trajectories, and an exploratory two-stage monitor on that corpus. On 235 clips from 15 scenario families, the two-stage protocol with a fixed T-2.5 s window obtains 83.0% accuracy and anomaly recall 0.773, versus 74.5% and 0.536 for a single-stage video-only baseline; specificity is 0.880 versus 0.928.
| Comments: | 7 pages. Submitted to IEEE ICRA 2027 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07540 [cs.CV] |
| (or arXiv:2611.07540v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07540 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aliasghar Arab [view email]
[v1]
Thu, 5 Nov 2026 06:48:39 UTC (3,454 KB)
来源：arXiv cs.AI · arxiv.org