跳到正文
arXiv cs.AI· Zecheng Hao, Zihan Huang, Shenghao Xie, Jiyuan Zhang, Zhaofei Yu, Tiejun Huang·· 1 小时前AI 评分33

AdaS-ViT:测试时自适应路由的脉冲视觉 Transformer,实现超高能效

AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency

AI 导读

研究者提出测试时自适应路由架构 AdaS-ViT,核心是 AdaE-IF 模型,根据脉冲 token 的注意力分数将显著区域路由到细粒度发放模式,其余区域采用粗粒度节能模式,并配合 S-GSA 和 S-LGU 模块在 token 与通道粒度选择性发放脉冲。在 ImageNet-1k 上每张图像仅耗电 1.02 至 2.45 毫焦,准确率-能耗比达 32× 至 72×。

正文 · 原文

View PDF HTML (experimental)

Abstract:Spiking Neural Networks (SNNs) promise event-driven and energy-efficient vision, yet existing spiking vision transformers (S-ViTs) largely confine spiking tokens to fixed and uniform firing schemes along the temporal, spatial and channel dimensions, whereas suppressing the computational redundancy thus incurred calls for adaptive firing mechanisms that operate jointly across all three axes at test time. To this end, we propose the Adaptive Efficient Integrate-and-Fire (AdaE-IF) model, which consults the attention score of each spiking token and routes salient regions to fine-grained firing patterns while assigning energy-frugal coarse patterns to the remaining regions. Building on AdaE-IF, we further introduce Spiking Gated Self-Attention (S-GSA) and Spiking Local-Gated Unit (S-LGU), which enforce selective spike emission at token and channel granularity. We integrate the aforementioned mechanisms and modules into AdaS-ViT, a test-time adaptive-routing architecture trained with a progressive spatio-temporal curriculum that stabilizes the co-adaptation of firing precision and routing masks. Extensive experiments across benchmarks spanning diverse data scales and modalities demonstrate that AdaS-ViT consistently achieves highly competitive performance. Notably, on ImageNet-1k it consumes only 1.02 to 2.45 millijoules per image and attains an accuracy-per-energy ratio of 32$\times$ to 72$\times$, operating in an economical inference regime and thereby pushing the energy-efficiency boundary further.
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2611.23728 [cs.NE]
  (or arXiv:2611.23728v1 [cs.NE] for this version)
  https://doi.org/10.48550/arXiv.2611.23728

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zecheng Hao [view email]
[v1] Wed, 18 Nov 2026 02:40:45 UTC (462 KB)

来源:arXiv cs.AI · arxiv.org