跳到正文
arXiv cs.AI· Ziv Kfir, Ido Kfir·· 56 分钟前AI 评分36

KV-LTM:以会话级持久 KV cache 作为个人助理的长期记忆

KV-LTM: Persistent Per-Session KV Caches as Long-Term Memory for Personal Assistants

AI 导读

arXiv 论文提出 KV-LTM,将每个会话的原始文本与其 KV cache 存为记忆层,回答时恢复缓存会话并附加问题,避免重复 prefill 计算。在 LongMemEval 的系统级对比中,两款商业助理得分低于同一模型离线读历史。分析显示每 token 状态约 31K-328K 字节,按每天 30K token 计每用户每年约 0.34-3.6 TB 存储开销。

正文 · 原文

View PDF HTML (experimental)

Abstract:Personal assistants accumulate months of conversations, but today's long-term memory stacks put derivation or selection between a question and its evidence. In a system-level comparison on LongMemEval, two commercial assistants scored well below the same model reading the history offline [1], consistent with, but not isolating, content loss between write and read. We propose KV-LTM: a memory tier storing each session's raw text plus its KV cache under a fixed reader model, answering by restoring cached sessions and appending the question. KV-LTM is complementary to, not a replacement for, retrieval and indexing. Text is the memory of record; the cache is a recompute-free prefill accelerator, degrading gracefully to text. We specify the design: storage record, budget-derived retention, append-only read path, and bounded aggregation. We size it analytically: across pinned BF16 full-attention (GQA) and latent-cached MLA configurations, per-token state spans approximately 31K-328K bytes (the pinned KDA+MLA hybrid differs: approximately 8.1K bytes/token plus a fixed per-session recurrent term), giving approximately 0.34-3.6 TB per user-year at a declared 30K-token/day scenario, against approximately 49 MB of raw text; per-query cost is modelled as regime-conditional transfer, attention, decode, aggregation, and compute. We define an evaluation protocol needing no persistent-cache implementation for logical QA evaluation: per-session prefill emulates valid cache reuse under the same model/input-state contract and ideal arithmetic, with named arms, controlled baselines, and a preregistered reader-model gate; a separate minimal real-cache replay tests numerical equivalence and measures the cost axis, with outcomes not assumed. KV-LTM trades storage for avoided re-prefill compute and write-time fidelity, within a declared retention budget.
Comments: 45 pages
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21998 [cs.AI]
  (or arXiv:2611.21998v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21998

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ziv Kfir [view email]
[v1] Mon, 16 Nov 2026 23:30:00 UTC (41 KB)

来源:arXiv cs.AI · arxiv.org