NeurIPS 2026 论文提出永久与瞬态表征解决持续强化学习的稳定性-可塑性权衡
Permanent and Transient Representations for Continual Reinforcement Learning
AI 导读
一篇 NeurIPS 2026 论文针对持续强化学习（CRL）的稳定性-可塑性权衡，为互补学习系统引入不同表征：长期学习用参数化表征、瞬态适应用非参数化表征。作者提出一种新的非参数化近似来估计瞬态价值函数，支持在线学习，在图像任务和 Craftax-Classic 上优于竞争基线。
正文 · 原文
Abstract:Continual Reinforcement Learning (CRL) agents struggle to adapt to new situations while retaining prior knowledge, leading to a stability-plasticity trade-off. One approach to solving this problem is by using complementary learning systems: one for slow, long-term learning and another for quick, transient adaptation. Anand & Precup (2023) instantiated this idea using permanent and transient value functions, but used the same features in both systems. Intuitively, long-term learning should rely more on parametric representations for broad generalization, while transient learning would be best served by non-parametric representations for situation-specific adaptation. In this paper, we explore this idea both theoretically and empirically. By introducing a novel non-parametric approximation for estimating the transient value function in complex tasks, we demonstrate that our method enables online learning and outperforms competitive baselines on image-based tasks and Craftax-Classic, underscoring the effectiveness of our system-level decomposition.
| Comments: | Accepted at the 40th Conference on Neural Information Processing Systems (NeurIPS 2026) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07327 [cs.LG] |
| (or arXiv:2611.07327v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07327 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nishanth Anand [view email]
[v1]
Thu, 5 Nov 2026 02:11:47 UTC (17,635 KB)
来源：arXiv cs.AI · arxiv.org