arXiv cs.AI· Zhen Liu, Zhiqin Yang, Yonggang Zhang, Jianyi Liu, Jinjun Wang, Jingwen Fu·· 3 小时前AI 评分35
AutoQS：用大语言模型自动化设计量化策略
AutoQS: Automated Quantization Strategy Design with Large Language Models
AI 导读
AutoQS 是一种借助大语言模型自动搜索量化策略的方法，在固定权重精度下按层和算子搜索 group-size 分配，用低成本质量代理和少量实际量化反馈在有限评估预算内迭代优化。在 Qwen2.5-7B-Instruct 上的四比特权重实验显示，其七任务平均精度较 uniform-group GPTQ 和 AWQ 分别提升 0.70 和 1.12 个百分点。
正文 · 原文
Abstract:Quantization reduces LLM deployment costs, but effective strategies must balance model quality and storage. LLM-driven search can automate strategy design through repeated proposal and evaluation. This process must identify useful strategies within a limited evaluation budget. The key challenge is that \textit{actual quantization is expensive, while inexpensive proxies may not reflect quantized-model performance.} To address this challenge, we present AutoQS, an \textbf{Auto}mated \textbf{Q}uantization \textbf{S}trategy Design with LLMs. AutoQS keeps weight precision fixed and searches group-size allocations across layers and operators under deployment constraints. Evaluation history guides coordinated LLM proposals, while deterministic validation ensures compatible allocations. An inexpensive quality proxy evaluates candidates, and an candidate repository retains alternative allocations to inform subsequent proposals. Periodic actual quantization complements these proxy evaluations with measured quality and checkpoint storage, providing feedback for further search within the evaluation budget. Experiments on Qwen2.5-7B-Instruct show that AutoQS improves seven-task average accuracy over uniform-group GPTQ and AWQ by 0.70 and 1.12 percentage points, respectively, at four-bit weight precision.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.11975 [cs.AI] |
| (or arXiv:2611.11975v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.11975 arXiv-issued DOI via DataCite |
Submission history
From: Zhen Liu [view email]
[v1]
Mon, 9 Nov 2026 06:59:18 UTC (185 KB)
[v2]
Wed, 11 Nov 2026 05:23:20 UTC (185 KB)
来源：arXiv cs.AI · arxiv.org