XLand-MiniGrid 记忆优势审计：LSTM 的领先究竟是记忆还是时钟？
Memory or a Clock? Auditing a Memory Advantage in XLand-MiniGrid with Seeds, Maps and Evaluation Noise Apart
AI 导读
一项研究审计了 XLand-MiniGrid 部分可观测任务中 LSTM 相对无记忆 MLP 的 25 个百分点优势。分离 60 次运行中的训练种子、地图与评估随机性后发现，“任务习得”判定 76% 是评估噪声，检出 10 分差距约需十个配对种子。在封存的新地图上，给 MLP 时间步后其与 LSTM 差距缩小到 3.3 分以内，表明大部分记忆优势实为知道时间的优势，并建议给无记忆基线提供时间步。
正文 · 原文
Abstract:On a partially observable XLand-MiniGrid task, an LSTM agent beats a memory-free MLP by 25 percentage points. The gap looks like a clear case of memory. We ask two questions about it. First, how much of a measured gap is noise? Separating training seed, map and evaluation randomness in 60 runs, each re-evaluated three times, we find that about ten paired seeds detect a 10-point gap in final success. The binary "task acquired" verdict, a common summary of learning curves, is 76% evaluation noise and would need about a hundred seeds for the same gap. Second, is the gap memory? On a sealed panel of new maps, opened once, a memory-free MLP of the LSTM's size comes within 3.3 points of the LSTM when it is given the time step, and gains no measurable benefit from its width without it. With the time step, an MLP a quarter of the LSTM's size, and the LSTM with its state erased at every step, also come within about 5 points. A linear probe reads the time step from the trained LSTM's state. On this task, most of the apparent memory advantage is an advantage of knowing the time, which matters because episodes are cut at a fixed limit and the reward shrinks with time. We recommend giving memory-free baselines the time step, and we release the protocol, the event-driven tooling and every checkpoint.
| Comments: | 17 pages, 7 figures. Code: this https URL . Data and checkpoints: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33816 [cs.LG] |
| (or arXiv:2610.33816v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33816 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rodrigo Reveron-Fonseca [view email]
[v1]
Sat, 24 Oct 2026 17:28:20 UTC (117 KB)
来源：arXiv cs.AI · arxiv.org