世界模型的上下文压缩研究
Context Compaction for World Models
AI 导读
一篇 arXiv 论文把世界模型重访区域时无法一致重建的问题视为上下文检索问题，并作出三项贡献：构建每次生成新世界以防止记忆训练数据的基准、开发首个针对世界模型做上下文压缩的 agent harness、用它研究四种世界模型设计。结果显示上下文压缩能让模型在上下文窗口之外最多 4 倍距离检索房间而无需重训练，而滑动窗口注意力模型在 dreaming 时丢失大部分检索能力。
正文 · 原文
Abstract:World models let observers explore and interact with generated worlds. Recent work shows that when observers revisit a region, current models often fail to regenerate it consistently. We view this as a context retrieval problem. LLM agents manage context retrieval at two layers of their stack. At the model layer, the KV cache design determines how much context fits in memory. At the agent harness layer, harnesses such as Claude Code pick the context sent to the LLM. Prior work on world models has mostly focused only on the model layer. In this paper, we make three contributions. First, we build a benchmark to measure world model context retrieval. It generates new worlds for every episode, preventing a model from recalling rooms using training data. It also separates context retrieval from guesses based on nearby frames. Second, we build an agent harness that performs context compaction for world models, which to our knowledge no prior work has studied. Third, we use our benchmark to study four world model designs. Context compaction lets our world models retrieve rooms up to 4 times beyond their context window without retraining. However, world models with sliding-window attention lose most of their context retrieval when dreaming. Therefore, a dreaming world agent benefits more from extending the reach of its world model using context compaction than switching to sliding window attention at the model layer.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.38980 [cs.AI] |
| (or arXiv:2610.38980v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38980 arXiv-issued DOI via DataCite |
Submission history
From: Hayder Tirmazi [view email]
[v1]
Wed, 28 Oct 2026 22:04:30 UTC (312 KB)
来源：arXiv cs.AI · arxiv.org