冻结记忆还不够：Engram 跨模型外部记忆作为提取机制的重新思考
Frozen Memory Is Not Enough: Rethinking External Memory as Extraction
AI 导读
一项 arXiv 研究通过跨模型冻结记忆提取实验，将源模型训练的 Engram 风格哈希记忆冻结后挂载到不同目标模型，仅训练轻量读取器，发现读取器与目标模型的对齐程度决定可用信号量。下游问答任务中，双层四分支读取器平均得分 38.8，几乎追平同模型复用效果；若提供方读取器与目标接口直接兼容，冻结记忆无需目标端训练即可提供可观效用。
正文 · 原文
Abstract:Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer. Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader? We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, while the amount of usable signal depends on how well the reader is aligned to the target model. In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement. These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.17050 [cs.CL] |
| (or arXiv:2608.17050v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.17050 arXiv-issued DOI via DataCite |
Submission history
From: Mingyuan Li [view email]
[v1]
Mon, 17 Aug 2026 18:54:42 UTC (568 KB)
[v2]
Wed, 19 Aug 2026 07:30:09 UTC (568 KB)
[v3]
Wed, 30 Sep 2026 07:57:51 UTC (565 KB)
[v4]
Tue, 13 Oct 2026 18:08:29 UTC (1,035 KB)
[v5]
Sat, 24 Oct 2026 14:21:46 UTC (689 KB)
来源：arXiv cs.AI · arxiv.org