arXiv cs.AI· Yuanhong Zheng, Ruichuan An, Xiaopeng Lin, Yuxing Liu, Sihan Yang, Huanyu Zhang, Haodong Li, Qintong Zhang, Renrui Zhang, Guopeng Li, Yifan Zhang, Yuheng Li, Wentao Zhang·· 2 小时前AI 评分42
PEARL:基于运行时定义概念的个性化流式视频理解
PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts
AI 导读
研究者提出个性化流式视频理解任务 PSVU,并发布首个基准 PSVU-Bench(标注 132 条视频、含时间戳问题)及即插即用基线 PEARL。不训练的情况下,PEARL 应用于三种架构,Real-Time QA 平均提升 15.05%、Past-Time QA 平均提升 12.60%;在 Qwen3-VL-8B 上训练后分别再提升 8.66% 和 9.39%。代码、数据与模型已开源。
正文 · 原文
Authors:Yuanhong Zheng, Ruichuan An, Xiaopeng Lin, Yuxing Liu, Sihan Yang, Huanyu Zhang, Haodong Li, Qintong Zhang, Renrui Zhang, Guopeng Li, Yifan Zhang, Yuheng Li, Wentao Zhang
Abstract:Unlike humans, who learn new concepts from a continuous visual stream, existing personalized vision-language models (VLMs) internalize a predefined concept set during training and operate on static images or offline videos. They cannot register a concept that a user defines in the middle of a stream, nor recall the earlier moments in which it appeared. We formalize this task as Personalized Streaming Video Understanding (PSVU) and introduce PSVU-Bench, the first benchmark for this task, which annotates 132 videos with timestamped questions. Its Concept-Definition, Real-Time and Past-Time QA ask a model to register concepts at arbitrary timestamps, recognize their current state, including user-defined actions, and retrieve their historical evidence. We propose PEARL, a strong plug-and-play baseline that stores user-defined concepts apart from archived video clips and rewrites each query with concept descriptions to retrieve the relevant history. Grounding a named concept in video is a skill shared by all concepts. We therefore construct PSVU-10K, a training set of verified concept questions on public internet videos, and train the retriever and answering VLM of PEARL on it once. Concepts defined at test time still require no training. Applied to three architectures without training, PEARL improves Real-Time QA by 15.05% and Past-Time QA by 12.60% on average over the corresponding offline models, and all three surpass existing online video models on both QA types. Training further improves PEARL on Qwen3-VL-8B by 8.66% on Real-Time QA and 9.39% on Past-Time QA, with gains in every category. Code, data and models are available at this https URL.
| Comments: | Arxiv Submission v2 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2603.20422 [cs.CV] |
| (or arXiv:2603.20422v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2603.20422 arXiv-issued DOI via DataCite |
Submission history
From: Yuanhong Zheng [view email]
[v1]
Fri, 20 Mar 2026 18:47:54 UTC (8,786 KB)
[v2]
Wed, 18 Nov 2026 09:30:33 UTC (16,683 KB)
来源:arXiv cs.AI · arxiv.org