arXiv cs.AI· Zhang Ziyue, Juwei Jian, Baiyang Song, Yunshan Zhong, Jun Peng, Yiyi Zhou, Rongrong Ji·· 1 小时前AI 评分37
NullTok:面向免训练长视频理解的经验零假设 token 选择方法
NullTok: empirical-null token selection for training-free long video understanding
AI 导读
NullTok 是一种免训练的长视频理解方法,通过为每个图像块基于 64 个无关对照查询构建经验零假设分布,将目标相关性校准为蒙特卡洛 p 值来筛选视觉 token。在 64 FFE 预算、无字幕无音频条件下,该方法在 Video-MME 上用冻结 LLaVA-Video-7B 和 Qwen3-VL-8B 骨干分别取得 65.81% 和 68.85%,超过同预算下所有被评测基线。代码已开源。
正文 · 原文
Abstract:Long video understanding remains a daunting challenge for existing multimodal large language models (MLLMs), mainly due to the large number of visual tokens accumulated in long videos. Currently, frame selection and token selection are two main paradigms for addressing this issue, both aiming to select relevant visual information to form a compact input for MLLMs. A natural approach combines these two paradigms by selecting frames before pruning tokens, but noisy frame-level relevance estimates can cause irreversible evidence loss. In this paper, we study this problem from the perspective of assessing the spatiotemporal distribution of query-specific evidence and propose a novel and training-free approach termed NULLTOK. For every patch across all frames, we construct an empirical-null distribution from 64 group-matched irrelevant control queries, capturing the scores expected in the absence of target-specific evidence. Target affinities are calibrated against these same-position references into right-tail Monte Carlo p-values, which are discrete super-uniform under a conditional exchangeability assumption. Calibrated screening and a high-recall semantic guard produce candidate tokens for deterministic visual deduplication and joint spatiotemporal budget allocation. To validate NullTok, we apply it to two representative MLLMs and conduct extensive experiments on a set of video benchmarks. The experimental results demonstrate its effectiveness under a bounded visual token budget. For example, it achieves 65.81% and 68.85% on Video-MME using frozen LLaVA-Video-7B and Qwen3-VL-8B backbones, respectively, at a cap of 64 full-frame equivalents (FFE), without subtitles or audio. These results outperform all evaluated baselines on Video-MME under the same budget for both backbones. Code is available at this https URL.
| Comments: | 8 pages |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23881 [cs.CV] |
| (or arXiv:2611.23881v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23881 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziyue Zhang [view email]
[v1]
Wed, 18 Nov 2026 05:32:52 UTC (2,619 KB)
来源:arXiv cs.AI · arxiv.org