arXiv cs.AI· Pengxiang Zhao, Xing Li, Hui-Ling Zhen, Weizhe Lin, Xianzhi Yu, Zhenhua Dong·· 3 小时前AI 评分29
SoScale：面向 FP4 量化的二阶尺度搜索方法
SoScale: Second-Order Scale Search for FP4 Quantization
AI 导读
arXiv 论文提出 SoScale，一种面向层级化 FP4 量化的二阶权重尺度码搜索方法。该方法在 Hessian 感知的层重建目标下搜索硬件有效的尺度码，并结合 GPTQ 式误差恢复，按对层输出的影响选择尺度。在 Qwen3 模型上，SoScale 在 MXFP4、NVFP4 和 HiF4 三种格式下均持续降低下游量化性能损失。
正文 · 原文
Abstract:Low-bit quantization reduces the memory footprint and inference cost of large language models (LLMs), yet accurate FP4 quantization remains challenging. Practical formats such as MXFP4, NVFP4, and HiF4 pair a 4-bit floating-point payload with block-wise or hierarchical scale codes, making accuracy highly sensitive to how each local distribution is scaled. Default maximum-absolute-value scaling preserves dynamic range, but can spend scarce FP4 levels on rarely occupied outlier regions and leave dense tensor values poorly resolved. Existing scale-search methods minimize local element-wise reconstruction error, optimizing tensor fidelity rather than layer-output fidelity. We propose SoScale, a second-order weight scale-code search method for hierarchical FP4 quantization. SoScale searches hardware-valid scale codes under a Hessian-aware layer reconstruction objective and couples this search with GPTQ-style error recovery, selecting scales by their impact on layer outputs. Across Qwen3 models, SoScale consistently reduces downstream quantization degradation under MXFP4, NVFP4, and HiF4.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.16426 [cs.LG] |
| (or arXiv:2611.16426v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16426 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pengxiang Zhao [view email]
[v1]
Thu, 12 Nov 2026 07:36:46 UTC (69 KB)
来源：arXiv cs.AI · arxiv.org