TraceKV：面向推理视觉语言模型的策略保持低秩 KV cache 压缩
TraceKV: Policy-Preserving Low-Rank KV Cache Compression for Reasoning Vision-Language Models
AI 导读
TraceKV 是一个免训练的低秩 KV cache 压缩框架，统一了 on-policy 校准、敏感度感知 SVD 和全局秩分配，显著降低推理过程中的 policy-KL 散度。在多个 VLM 与视觉推理基准上，结合量化可实现 5.64 倍 KV cache 压缩且推理精度近乎无损，优化的低秩注意力内核带来最高 1.83 倍的端到端解码加速。代码即将开源。
正文 · 原文
Abstract:Reasoning vision-language models (VLMs) impose substantial KV cache demands from both large multimodal contexts and growing reasoning traces. Low-rank KV compression has emerged as a promising paradigm that stores low-dimensional latents of KV states and can be combined with quantization. However, existing low-rank methods often induce growing predictive policy drift from the dense model when compressing reasoning KV states, resulting in reasoning failures. We identify two key causes of this degradation: calibration-state mismatch between static contexts and on-policy reasoning states, and unequal error sensitivity across KV states. To address these challenges, we propose TraceKV, a training-free low-rank KV compression framework that unifies on-policy calibration, sensitivity-aware SVD, and global rank allocation under a single policy-preserving principle, substantially reducing policy-KL divergence during reasoning. Experiments across multiple VLMs and visual reasoning benchmarks demonstrate superior accuracy--memory trade-offs, achieving near-lossless reasoning accuracy with 5.64$\times$ KV cache compression when combined with quantization. Optimized low-rank attention kernels further deliver up to 1.83$\times$ end-to-end decoding speedup. Code will be available at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.15569 [cs.CV] |
| (or arXiv:2611.15569v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15569 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ke Li [view email]
[v1]
Wed, 11 Nov 2026 14:54:19 UTC (1,272 KB)
来源：arXiv cs.AI · arxiv.org