用一个量解决低成本检索与观测信用分配：信息价值
Cheap Retrieval and Observation Credit Assignment from One Quantity: The Value of Information
AI 导读
一篇被 CoRL 2026 机器人基础模型记忆 Workshop 接收的论文提出，强化学习记忆中的检索成本与信用分配可由同一个量——信息价值——统一刻画，并通过学习到的世界模型从存档 rollout 中估计。该量在读观测时给出获取动作最优值的可证下界，从而抬高 TD target 并归因到相应观测；在读决策时定义最优检索集。基于此信号的试点检索系统在更短的上下文下完成了一个记忆任务。
正文 · 原文
Abstract:Memory in reinforcement learning raises two problems. Attention finds the relevant item by scoring the whole store, which is costly for robots under latency and energy limits. When a retrieved memory pays off, credit for acquiring it must cross the whole sequence. We argue that both are governed by one quantity, the value of information, and propose to estimate it from archived rollouts with a learned world model. We obtain that, when read at the observation, it yields a provable lower bound on the optimal value of the acquiring action, which floors its TD target and attributes this floor to the observation that earned it. Read at the decision, it defines the optimal retrieval set. A pilot retrieval system, to be trained and improved with this signal, solves a memory task with reduced context.
| Comments: | 11 pages, 1 figure. Accepted at the CoRL 2026 Workshop on Memory for Robot Foundation Models (this https URL) |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| ACM classes: | I.2.6; I.2.8 |
| Cite as: | arXiv:2610.40030 [cs.AI] |
| (or arXiv:2610.40030v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40030 arXiv-issued DOI via DataCite |
Submission history
From: Daniel Strauß [view email]
[v1]
Thu, 29 Oct 2026 13:36:06 UTC (26 KB)
来源：arXiv cs.AI · arxiv.org