SkimPrefill:让小模型先读一遍,加速长图文交错输入的 prefill
Let a Small Model Read First: Faster Prefill for Long Interleaved Image-Text Inputs
AI 导读
arXiv 论文提出 SkimPrefill 方法:先用小混合模型(Qwen3.5-0.8B)以半分辨率粗读最多 128K token 的长图文交错输入,用早期层单个注意力头定位与问题相关的单元,再把筛选后的 token 作为较短提示词交给目标模型。
正文 · 原文
Abstract:Vision-language models (VLMs) can answer questions about long interleaved inputs, such as documents of many pages and collections of hundreds of images, but they are slow to start: on inputs of about 100K tokens, the prefill takes most of the wait for the first output token. A common approach prunes the input by scoring its tokens with the target model itself or with a small model of the same family that first writes an answer. However, this scoring takes much of the time that pruning saves. We find that the evidence for a question lies in a few whole units (images or paragraphs), and that a small hybrid model finds them with one attention head of an early layer, even at half image resolution. Based on these findings, we propose SkimPrefill, which lets the small hybrid model read the whole input first, up to that early layer, with images at half resolution. Most layers of the small hybrid model (Qwen3.5-0.8B) run in linear time, so this pass is short. SkimPrefill then ranks every unit with that head and gives the target the top units as an ordinary shorter prompt. One untrained selector serves targets of different model families. Extensive experiments on MMLongBench at 128K tokens, with 19 datasets of five task types, show the effectiveness of SkimPrefill. With 10% of the input tokens kept, SkimPrefill speeds up the prefill of Qwen3.5-9B, Gemma-4-12B and Qwen3-VL-32B by 3.7 times, 11.6 time and 8.2 times while maintaining their average accuracy.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24431 [cs.CV] |
| (or arXiv:2611.24431v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24431 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Grace Li Zhang [view email]
[v1]
Wed, 18 Nov 2026 13:49:02 UTC (1,265 KB)
来源:arXiv cs.AI · arxiv.org