ResCom：用激活残差补偿解决大语言模型低比特量化精度损失
ResCom: Activation Residual Compensation for Large Language Model Quantization
AI 导读
arXiv 论文提出 ResCom，一种利用激活舍入残差的低秩校正方法，补偿大语言模型低比特权重-激活量化带来的层输出误差，无需梯度训练或更新已校准权重。
正文 · 原文
Abstract:Low-bit weight--activation quantization reduces the memory and computation needed to deploy large language models, but activation rounding can still degrade accuracy after calibration. Reconstruction from rounded inputs cannot distinguish activations within the same quantization bin. We propose ResCom, a low-rank correction that uses the activation rounding residual to compensate for the resulting layer-output error. An output-aware calibration procedure selects residual directions by their contribution to output reconstruction and computes the correction in closed form, without gradient training or updates to the calibrated weights. On Llama-3-8B with 4-bit weights and activations, rank-32 attention corrections reduce WikiText-2 perplexity from 7.708 to 7.147, improve MMLU by 2.89 percentage points, and raise five-shot GSM8K accuracy from 23.96 to 36.16 percent. Evaluations on models from 1B to 70B parameters and on held-out C4 test the method beyond this setting. The evaluated FP32 implementation adds 2.25 percent to 4K prefill time; a separate compact-storage implementation retains a 27.7 percent peak-memory reduction relative to FP16.
| Comments: | 25 pages (10 pages main text + appendix), 9 figures, 20 tables |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.6; I.2.7 |
| Cite as: | arXiv:2610.35955 [cs.LG] |
| (or arXiv:2610.35955v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35955 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ning Yang [view email]
[v1]
Sun, 25 Oct 2026 03:02:21 UTC (249 KB)
来源：arXiv cs.AI · arxiv.org