PPS：即插即用的扫视视觉方法用于细粒度分类
PPS: Plug-and-Play Saccadic Vision for Fine-Grained Classification
AI 导读
PPS 是一种受人类扫视视觉启发的轻量级扩展，可将任意带空间特征图的预训练 backbone 变成基于 glimpse 推理的细粒度视觉分类器，仅需添加一个小的 priority head 和单个 cross-attention 融合块。
正文 · 原文
Abstract:Fine-grained visual classification benefits from high-resolution inputs to resolve subtle, localized cues, but at the cost of large compute and memory budgets. Inspired by human saccadic vision, we propose a lightweight extension that turns any pretrained backbone with spatial feature maps into a saccadic classifier through glimpse-induced inference. The only architectural additions are a small priority head and a single cross-attention fusion block. The same weight-tied encoder encodes a downsampled peripheral glance, exposes the spatial feature map from which the priority head predicts a glimpse-priority distribution and re-encodes high-resolution glimpse crops sampled from that distribution in parallel. A sequential sampler with non-maximum suppression draws a runtime-configurable number of glimpses. A residual multi-head cross-attention block then fuses the glance against the glimpses. To improve the quality of the priority maps, we train the priority head by a per-glimpse information-gain target from a lightweight per-image memory bank. This requires no reinforcement learning. Complexity is linear in the number of glimpses and peak memory is decoupled from input resolution. Across nine benchmarks our framework improves consistently over its backbones.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2509.15688 [cs.CV] |
| (or arXiv:2509.15688v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2509.15688 arXiv-issued DOI via DataCite |
Submission history
From: Johann Schmidt johSchm [view email]
[v1]
Fri, 19 Sep 2025 07:03:37 UTC (11,385 KB)
[v2]
Thu, 29 Oct 2026 20:00:22 UTC (14,978 KB)
来源：arXiv cs.AI · arxiv.org