arXiv cs.AI· Daniele Solombrino, Antonio Andrea Gargiulo, Alessandro Zirilli, Luca Zhou, Adrian Robert Minut, Emanuele Rodol\`a·· 4 小时前AI 评分48
零样本量化新方法：通过权重空间算术提取"量化向量"
Zero-Shot Quantization via Weight-Space Arithmetic
AI 导读
一篇 NeurIPS 2026 论文提出"量化向量"方法：从 donor 任务通过权重空间算术提取对后训练量化（PTQ）鲁棒的方向，用以修补 receiver 模型，在 3-bit 设置下将 PTQ Top-1 准确率最高提升 60%，且无需接收侧的量化感知训练（QAT）。
正文 · 原文
Abstract:We show that robustness to post-training quantization (PTQ) is a transferable direction in weight space. We call this direction the quantization vector: extracted from a donor task by simple weight-space arithmetic, it can be used to patch a receiver model and improve PTQ Top-1 accuracy by up to 60% in a 3-bit setting, without receiver-side quantization-aware training (QAT). When using no receiver validation data, the method provides a zero-shot, low-cost alternative to QAT for extremely low-bit deployment. Across multiple vision and language models and more than 30 tasks, donor quantization vectors often yield substantial gains even when donor and receiver tasks differ markedly. We further prove rigorously that quantization vectors are well-defined and do not suffer from reparameterization symmetries, and provide a local geometric account of their effect. Together, these results suggest that quantization robustness can be partially isolated, reused, and transferred through simple weight-space algebra.
| Comments: | Accepted at NeurIPS 2026. 40 pages. OpenReview: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.03420 [cs.CV] |
| (or arXiv:2604.03420v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2604.03420 arXiv-issued DOI via DataCite |
Submission history
From: Daniele Solombrino [view email]
[v1]
Fri, 3 Apr 2026 19:33:25 UTC (834 KB)
[v2]
Sat, 11 Apr 2026 19:49:30 UTC (844 KB)
[v3]
Fri, 8 May 2026 06:46:47 UTC (2,627 KB)
[v4]
Thu, 5 Nov 2026 14:09:30 UTC (1,913 KB)
来源：arXiv cs.AI · arxiv.org