LLM 智能体的记忆信任缺口:Qwen3 系列跨模型规模的过时证据依赖研究
The Memory Trust Gap: Reliance on Stale Evidence Across Model Scales in LLM Agents
AI 导读
一项研究考察了 LLM 智能体依赖过时记忆信息的问题,覆盖 Qwen3 0.6/1.7/4/8B 四个 checkpoint,采用 Benefit 与 Safety 两个动作评分测试套件。
正文 · 原文
Abstract:Persistent-memory agents can follow stale information despite current authoritative evidence. We study stale-evidence reliance across Qwen3 0.6/1.7/4/8B checkpoints using two controlled, action-scored suites: a Benefit suite requiring stored information and a Safety suite providing current evidence as static prompt text. We distinguish stale-value reliance from net accuracy loss relative to no memory. Benefit-suite reliance is 0.92-1.00 across checkpoints. In the Safety suite, larger checkpoints resist a plain stale note but incur substantial harm when it appears newer. A full $2\times2\times2\times2$ factorial manipulates label, recency, source authority, and position. Direct cross-size contrasts identify stronger recency effects in the larger tested checkpoints and a position-effect sign reversal within Qwen3. Metadata improves accuracy at all four checkpoints, with larger gains at 4B/8B; pre-resolution yields near-ceiling accuracy across all four. Llama-Instruct comparisons provide cross-family evidence for selected effects, while RGB and MisBench corroborate stale-value reliance. A framing control shows that the failure extends beyond memory-labeled content. These results characterize cue-dependent vulnerability across tested checkpoints without isolating capability or evaluating a complete memory lifecycle.
| Comments: | Accepted at the NeurIPS 2026 Workshop PALM: Personalized, Aligned, Long-Term Memory for AI Systems. 14 pages, 7 figures, 11 tables. v2: camera-ready version |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.01852 [cs.AI] |
| (or arXiv:2609.01852v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.01852 arXiv-issued DOI via DataCite |
Submission history
From: Jundong Hu [view email]
[v1]
Tue, 1 Sep 2026 20:35:00 UTC (114 KB)
[v2]
Mon, 16 Nov 2026 21:11:56 UTC (216 KB)
来源:arXiv cs.AI · arxiv.org