写入关键内容：循环记忆中的 Token 选择与槽位分配解耦
Write What Matters: Disentangling Token Selection from Slot Allocation in Recurrent Memory
AI 导读
研究提出 Factorized Write Routing（FWR），将循环记忆模型的写入决策拆分为先决定保留或跳过 token、再选择固定或自适应槽位数，查询仍读取全部槽位。
正文 · 原文
Abstract:Long-sequence models must retain earlier information. Full attention keeps all past tokens but grows expensive, and recurrent-memory models compress history into fixed slots for linear-time processing. Limited capacity makes writes consequential: distractors can interfere with useful content, while skipped evidence cannot be recalled. Existing multi-slot memories write every token to the same number of slots, merging whether to store it with how many slots to update. We introduce Factorized Write Routing (FWR), which first keeps or skips each token and then chooses a fixed or token-adaptive number of slots. Queries still read all slots. Across ten seeds on two synthetic tasks that query facts after long contexts, we observe a density-dependent reversal. With four slots per write, long-context accuracy favors always writing on dense Feature Recall (0.472 vs 0.337), but learned selective writing on sparse-fact Passkey (0.222 vs 0.048). The tested adaptive rule underperforms fixed allocation under the learned gate on both tasks. A diagnostic oracle given fact labels unavailable at deployment reaches 0.372 on Passkey, showing headroom in filtering distractors. The results suggest avoiding missed writes in information-dense contexts and filtering distractors when they dominate.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.34180 [cs.CL] |
| (or arXiv:2610.34180v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34180 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chen Chen [view email]
[v1]
Sun, 25 Oct 2026 09:33:54 UTC (447 KB)
来源：arXiv cs.AI · arxiv.org