arXiv cs.AI· Junxiang Wen, Yinxi Zhao, Jiawei Chen, Jingying Hu, Lijun Wang, Zhen Qin, Shuiguang Deng·· 2 小时前AI 评分35
APS:面向长视频理解的基于自适应分区的帧选择框架
APS: Adaptive Partitioning-Based Frame Selection for Long Video Understanding
AI 导读
研究者提出免训练的帧选择框架 APS,通过把时间分区建模为复杂度正则化的拟合问题,用动态规划联合优化分区边界与局部模型,使分区随 query 与帧的相关性自适应变化。在 Video-MME、MLVU 和 LongVideoBench 上的实验表明,APS 在多种 LVLM 骨干上一致提升性能,相对均匀采样取得同类方法中最高的平均增益,并具备跨模型规模与相似度编码器的泛化性。
正文 · 原文
Abstract:Recent advances in large vision-language models (LVLMs) have significantly improved video understanding capabilities, yet processing long videos remains challenging due to the substantial computational cost of extensive visual inputs. Frame selection offers an efficient alternative, but many existing region-based methods rely on fixed or predefined temporal partitions, which may result in suboptimal temporal regions, compromising subsequent budget allocation and frame selection. In this work, we propose Adaptive Partitioning-Based Frame Selection (APS), a training-free framework that adapts temporal partitions to local variations in query-frame relevance. Specifically, APS formulates temporal partitioning as a complexity-regularized fitting problem, jointly optimizing partition boundaries and local models through dynamic programming. The resulting adaptive temporal partitions guide frame budget allocation based on query relevance and temporal extent, enabling informative frame selection under limited visual budgets. Extensive experiments on Video-MME, MLVU, and LongVideoBench demonstrate that APS consistently improves performance across diverse LVLM backbones, achieving the highest average gains over uniform sampling among competing methods. Further analyses validate the effectiveness of adaptive partitioning and demonstrate its generalizability across model scales, similarity encoders, and advanced LVLMs.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24079 [cs.CV] |
| (or arXiv:2611.24079v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24079 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Junxiang Wen [view email]
[v1]
Wed, 18 Nov 2026 09:07:28 UTC (15,321 KB)
来源:arXiv cs.AI · arxiv.org