KV-LTM:以会话级持久 KV cache 作为个人助理的长期记忆
KV-LTM: Persistent Per-Session KV Caches as Long-Term Memory for Personal Assistants
AI 导读
arXiv 论文提出 KV-LTM,将每个会话的原始文本与其 KV cache 存为记忆层,回答时恢复缓存会话并附加问题,避免重复 prefill 计算。在 LongMemEval 的系统级对比中,两款商业助理得分低于同一模型离线读历史。分析显示每 token 状态约 31K-328K 字节,按每天 30K token 计每用户每年约 0.34-3.6 TB 存储开销。
正文 · 原文
Abstract:Personal assistants accumulate months of conversations, but today's long-term memory stacks put derivation or selection between a question and its evidence. In a system-level comparison on LongMemEval, two commercial assistants scored well below the same model reading the history offline [1], consistent with, but not isolating, content loss between write and read. We propose KV-LTM: a memory tier storing each session's raw text plus its KV cache under a fixed reader model, answering by restoring cached sessions and appending the question. KV-LTM is complementary to, not a replacement for, retrieval and indexing. Text is the memory of record; the cache is a recompute-free prefill accelerator, degrading gracefully to text. We specify the design: storage record, budget-derived retention, append-only read path, and bounded aggregation. We size it analytically: across pinned BF16 full-attention (GQA) and latent-cached MLA configurations, per-token state spans approximately 31K-328K bytes (the pinned KDA+MLA hybrid differs: approximately 8.1K bytes/token plus a fixed per-session recurrent term), giving approximately 0.34-3.6 TB per user-year at a declared 30K-token/day scenario, against approximately 49 MB of raw text; per-query cost is modelled as regime-conditional transfer, attention, decode, aggregation, and compute. We define an evaluation protocol needing no persistent-cache implementation for logical QA evaluation: per-session prefill emulates valid cache reuse under the same model/input-state contract and ideal arithmetic, with named arms, controlled baselines, and a preregistered reader-model gate; a separate minimal real-cache replay tests numerical equivalence and measures the cost axis, with outcomes not assumed. KV-LTM trades storage for avoided re-prefill compute and write-time fidelity, within a declared retention budget.
| Comments: | 45 pages |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.21998 [cs.AI] |
| (or arXiv:2611.21998v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21998 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziv Kfir [view email]
[v1]
Mon, 16 Nov 2026 23:30:00 UTC (41 KB)
来源:arXiv cs.AI · arxiv.org