arXiv cs.AI· Seohyeon Cha, Huancheng Chen, Dongjun Kim, Haoran Zhang, Kevin Chan, Gustavo de Veciana, Haris Vikalo·· 2 小时前AI 评分37
CoreQ:面向 LLM 量化的失配校正逐轮量化方法
CoreQ: Mismatch-Corrected Successive Rounding for LLM Quantization
AI 导读
CoreQ 是一种免训练的大语言模型训练后量化(PTQ)框架,通过失配的几何分解得到闭式、逐层自适应的校正系数,缓解逐层量化带来的输入失配和校准数据过拟合问题,再用贪心逐轮量化求解三角最小二乘目标。
正文 · 原文
Abstract:Post-training quantization (PTQ) enables efficient deployment of large language models by mapping pretrained weights to low-bit formats without retraining, typically using a small calibration set to minimize a layer-wise calibration objective. However, this sequential procedure induces a mismatch: errors from earlier quantized layers alter the inputs received by later layers, causing the activations to deviate from those of the full-precision model. Recent approaches introduce mismatch-aware calibration objectives to compensate for this effect, but leave open how much of the observed mismatch should shift each layer's calibration target. We show that fully applying the empirical mismatch correction can overfit finite calibration data, while using a fixed correction coefficient ignores variation in mismatch estimates across layers. To address these limitations, we propose CoreQ, a learning-free PTQ framework that uses a geometric decomposition of the mismatch to compute a closed-form, layer-adaptive correction coefficient. This coefficient controls how much of the observed mismatch enters each layer's calibration target, reducing overfitting to limited calibration data without coefficient tuning. CoreQ then applies efficient greedy successive rounding to the resulting triangular least-squares objective. Its bounded beam-search extension, K-CoreQ, trades modest additional compute for improved performance. Across multiple LLM families, scales, bit-widths, and quantization settings, CoreQ and K-CoreQ achieve lower perplexity and higher downstream accuracy than strong PTQ baselines. Code is available at GitHub.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.05902 [cs.LG] |
| (or arXiv:2602.05902v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.05902 arXiv-issued DOI via DataCite |
|
| Journal reference: | NeurIPS 2026 |
Submission history
From: Seohyeon Cha [view email]
[v1]
Thu, 5 Feb 2026 17:18:02 UTC (459 KB)
[v2]
Fri, 8 May 2026 19:18:18 UTC (993 KB)
[v3]
Wed, 18 Nov 2026 14:10:52 UTC (1,005 KB)
来源:arXiv cs.AI · arxiv.org