arXiv cs.AI· Tianqi Chen, Xunchao Li, Linchuan Xie, Guanghua Yu, Jianchen Zhu, Peisong Wang, Jian Cheng·· 4 小时前AI 评分39
GateRQ：面向 LLM 量化条件记忆的逐查询精度方法
Let the Gate Decide: Per-Lookup Precision for Quantized Conditional Memory in LLMs
AI 导读
一篇 arXiv 论文提出 GateRQ，将 DeepSeek Engram 等条件记忆层的每行以 1-bit 残差码梯级存储，用 gate 决定哪些查询读取更深层级，无需重训练即可压缩记忆表。
正文 · 原文
Abstract:Conditional memory layers, such as DeepSeek's Engram, attach large hashed n-gram tables to a language model, and removing them from DeepSeek-V4.1-Flash costs 36.8 PopQA points. Low-bit quantization loses long-tail facts first, and the standard remedy, giving more bits to frequently accessed rows, backfires: refining lookups by access frequency is worse than refining them at random. We trace this to the gate that scales each retrieved row. A lookup's quantization error is weighted by its gate, which is small for frequent n-grams, large for the rare n-grams of long-tail entities, and computable from a 1-bit code. GateRQ therefore stores each row as a ladder of 1-bit residual codes, keeps level 1 in GPU memory to compute the gate, and reads deeper levels only for the lookups the gate ranks highest. Without retraining, GateRQ cuts GPU table memory by 7.8x on DeepSeek-V4.1-Flash, which raises vLLM throughput by 11%-31% over FP8 tables, while losing about 1 PopQA point at 1.50 bits read per lookup and 0.2 points at 2.10 bits across two models.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.35985 [cs.LG] |
| (or arXiv:2610.35985v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35985 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chen Tianqi [view email]
[v1]
Mon, 26 Oct 2026 11:53:58 UTC (401 KB)
来源：arXiv cs.AI · arxiv.org