arXiv cs.AI· Aditya Gupta, Jean Sebastien Dandurand, Kai Qiu, Rohan Choudhury, Laszlo A. Jeni·· 3 小时前AI 评分37
SPLIT：学习层级化图像分块策略，让 ViT 更快
Learning Hierarchical Patch Splitting Policies for Faster Vision Transformers
AI 导读
SPLIT（Split Policy Learned Image Tokenization）根据模型认为有意义的任务相关信息来分配 token 分辨率，替代以往基于局部熵等像素级启发式的自适应分块方法。
正文 · 原文
Abstract:Vision Transformers (ViTs) are bottlenecked by the length of their input token sequence. Standard patchification fixes this length with a single patch size applied across the image, regardless of regional content. Prior work on adaptive patch sizing reduces token count by assigning coarser patches to visually simple regions, but relies on noisy pixel-level heuristics such as local entropy to make patchification decisions. We argue that this compute allocation should be grounded in the model's interpretation of task relevant information, not in raw pixel statistics. We introduce Split Policy Learned Image Tokenization which learns to allocate token resolution based on what the model finds meaningful, producing shorter token sequences that are better matched to task-relevant content. SPLIT matches the accuracy of full-resolution ViT classification baselines while using 30% fewer tokens. The same gains hold on dense prediction: SPLIT achieves competitive accuracy on COCO and ADE20K, demonstrating that semantically aligned patchification generalises well beyond classification.
| Comments: | Accepted at NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.05742 [cs.CV] |
| (or arXiv:2611.05742v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05742 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aditya Gupta [view email]
[v1]
Tue, 3 Nov 2026 23:28:16 UTC (21,119 KB)
来源：arXiv cs.AI · arxiv.org