DCKV：混合语言模型中提示词 KV cache 的可学习变换编码

DCKV: Learned Transform Coding of the Prompt KV Cache in Hybrid Language Models

AI 导读

DCKV 是首个端到端训练的 KV cache 可微编解码器，用于压缩混合语言模型中仅存于少数全注意力层的提示词缓存，其变换、量化器与熵模型以提示词 token 和已解码邻位为条件，针对冻结语言模型联合优化。在 Qwen3.5-2B 上，DCKV 以超过 55 倍于 bf16 的压缩比存储每个 token，三个长上下文基准最多损失 1.8 分；压缩超 100 倍时最多损失 3.6 分。

正文 · 原文

View PDF HTML (experimental)

Abstract:Hybrid language models keep a key-value (KV) cache only in their few full-attention layers, but this cache is still the part of a prompt's state that grows with the prompt and that must be stored or moved to reuse it. We present DCKV, a differentiable codec for the KV cache and the first KV-cache codec that is trained end to end. Its transform, its quantizer and its entropy models, which are conditioned on the prompt tokens and on already decoded neighbouring positions, are optimized together against the frozen language model, both for the code length that the entropy models assign and for the change of the model's output. On Qwen3.5-2B, DCKV stores a compressed token in more than 55 times fewer bytes than bf16 at a loss of at most 1.8 points on three long-context benchmarks, and in more than 100 times fewer bytes at a loss of at most 3.6 points.

Comments:	21 pages, 7 figures, 14 tables
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.36950 [cs.LG]
	(or arXiv:2610.36950v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.36950 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Egor Maximov [view email]
[v1] Tue, 27 Oct 2026 10:11:04 UTC (296 KB)