arXiv cs.AI· Xianglong Wang, Neil Rodrigues, Pei Guo, Yiru Fang, Kevin Beussman, Shuyang Wang, Jingzhi Wang, Liangju Li, Shunyao Xu, Zhongwei Cheng, Wei Li, Lin Chen·· 2 小时前AI 评分38
OpenHome：面向智能家居摄像头视频从感知到洞察的评测基准
OpenHome: From Perception to Insight in Smart-Home Camera Videos
AI 导读
研究者推出 OpenHome，一个纯视觉的四选一智能家居视频理解评测基准，包含 1,118 段真实视频上的 2,254 道题，将 10 个任务组织为 Perception、Reasoning、Insight 三层能力。
正文 · 原文
Authors:Xianglong Wang, Neil Rodrigues, Pei Guo, Yiru Fang, Kevin Beussman, Shuyang Wang, Jingzhi Wang, Liangju Li, Shunyao Xu, Zhongwei Cheng, Wei Li, Lin Chen
Abstract:Vision language models (VLMs) have been increasingly adopted in consumer smart-home camera systems to interpret recorded videos and provide insights. However, video clips captured from smart-home cameras are different from curated web videos dominating existing VLM benchmarks. Smart-home video clips often do not contain subtitles or narration and are captured from different viewing angles, under various lighting conditions, and in diverse scenarios. However, the capabilities of the model in this domain, under sparse visual signals and limited context, are not fully assessed by existing VLM benchmarks. To close this gap, we introduce OpenHome, a visual-only, four-option multiple-choice benchmark for evaluating smart-home video understanding. It contains 2,254 questions on 1,118 real-world video clips and organizes 10 tasks into three capability levels. Perception evaluates visible content, Reasoning evaluates causal, spatial, and temporal relationships, and Insight evaluates subject intent, supporting evidence and appropriate homeowner responses. Based on the dataset, we describe the task design, data pipeline and evaluation protocol, and report results for 32 closed-source and open-weight VLMs. The best model, Gemini 4 Argon, reaches 81.2% overall accuracy. Frontier closed-source models outperform open-weight models on every level. The best closed-source model leads the best open-weight model by 16.3% on Perception, 22.4% on Reasoning and 37.4% on Insight.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.31986 [cs.CV] |
| (or arXiv:2610.31986v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31986 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei Li [view email]
[v1]
Thu, 22 Oct 2026 23:04:49 UTC (2,332 KB)
来源：arXiv cs.AI · arxiv.org