arXiv cs.AI· Rinyoichi Takezoe, Yaqian Li, Weiren Wang, Zihao Bo, Huihong Shi, Anzhou Hou, Mo Guang, Jiale Yan, Kaiwen Long·· 3 小时前AI 评分35
LearnPrunerV2：面向视觉语言模型的可学习渐进式视觉 token 剪枝
LearnPrunerV2: Learnable Progressive Token Pruning for Vision-Language Models
AI 导读
LearnPrunerV2 提出一种完全可学习的视觉 token 剪枝架构，通过在每个 VLM 层嵌入轻量模块、融合视觉显著性与 query 相关性来估计 token 重要性，并引入 Budget-Deterministic Pruning（BDP）设定固定的每层剪枝预算。在 LLaVA-v1.5 上剪去 77.8% 的视觉 token 时仍保留 99.8% 的原始性能，超越此前 SOTA 方法。
正文 · 原文
Abstract:Vision-Language Models (VLMs) have demonstrated remarkable multimodal capabilities but suffer from prohibitive computational overhead due to the massive number of visual tokens. While token pruning is a promising solution, existing methods still suffer from two major drawbacks: 1) unreliable pruning criteria that fail to accurately localize task-specific regions due to inherent attention biases or query-agnostic designs, and 2) suboptimal pruning strategies that employ rigid, non-adaptive schedules, overlooking the dynamic evolution of token redundancy across different model depths. To address these challenges, we propose LearnPrunerV2, which extends the previous two-stage heuristic framework (LearnPruner) into a fully learnable architecture. Specifically, we integrate lightweight learnable modules into every layer of the VLM backbone, which estimate token importance by fusing intrinsic visual saliency with extrinsic query relevance captured via a shadow token. By uncovering a consistent staircase redundancy pattern across VLM layers, we further introduce Budget-Deterministic Pruning (BDP) to establish a hardware-friendly schedule with fixed per-layer budgets. Extensive experimental results across various benchmarks and VLMs demonstrate that LearnPrunerV2 consistently outperform previous state-of-the-art methods. Notably, LearnPrunerV2 preserves 99.8% of the original performance when pruning 77.8% of visual tokens on LLaVA-v1.5, achieving a superior accuracy-efficiency trade-off.
| Comments: | Accepted to ACM Multimedia 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36986 [cs.CV] |
| (or arXiv:2610.36986v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36986 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rinyoichi Takezoe [view email]
[v1]
Tue, 27 Oct 2026 10:35:41 UTC (5,219 KB)
来源：arXiv cs.AI · arxiv.org