arXiv cs.AI· Oleeviya Babu Poikarayil, Tony Basil Babu, Abdulrahman Nahhas, Het Darshan Mehta, Jawid Ahmad Baktash, Ernesto William De Luca, Klaus Turowski·· 3 小时前AI 评分45
当检索还不够：Agent 记忆中的过时事实研究
When Retrieval Is Not Enough: Obsolete Facts in Agent Memory
AI 导读
arXiv 论文在 MemoryAgentBench 的 FactConsolidation 任务上分析记忆型 Agent 的失败来源：Qwen2.5-7B-Instruct 在 6K 历史 BM25 下，检索证据预算从 64 增至 512 token 时准确率从 45% 降到 32%。
正文 · 原文
Abstract:Memory-agent benchmarks often collapse retrieval and answer generation into one score, making it hard to tell whether a failure comes from missing an update or from not using an update that was already retrieved. We examine this distinction on the single-hop FactConsolidation task of MemoryAgentBench. With Qwen2.5-7B-Instruct on 6K-history BM25 accuracy drops from 45% to 32% when the retrieved evidence budget grows from 64 to 512 tokens, even though a gold-assisted post-hoc audit finds the newest statement in every retrieved set. Most of the errors occur on questions that also contain an obsolete statement version. Under a frozen prompt and equal four-statement evidence size, replacing one unrelated statement with the obsolete competing statement reduces accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B, with all 50 harmful paired flips classified as stale. A 32K-history replication likewise favors BM25-64 over BM25-512 (69% versus 58%). These results suggest that retrieving the correct fact is not enough. An obsolete competing statement can be more harmful than an unrelated piece of evidence of the same size.
| Comments: | Accepted at the NeurIPS 2026 Workshop on Personalized, Aligned, Long-Term Memory for AI Systems (PALM). 23 pages, including supplementary material |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36308 [cs.AI] |
| (or arXiv:2610.36308v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36308 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jawid Ahmad Baktash [view email]
[v1]
Mon, 26 Oct 2026 21:30:26 UTC (136 KB)
来源：arXiv cs.AI · arxiv.org