基于潜在状态编码的交互式强化学习中的广域持久性建议（Encoder BPA）
Broad-Persistent Advising in Interactive Reinforcement Learning via Latent State Encoding
AI 导读
一项 arXiv 论文提出 Encoder BPA，用训练好的编码器将状态序列映射为潜在表示，通过潜在比较、top-k 聚合和一致性条件检索并复用外部建议，改进传统 Deep IntRL 每条建议仅用一次的做法。实验显示 Encoder BPA 在训练早期比多种 Deep IntRL 方法提升学习效率和建议利用率，尤其在复杂和部分可观测环境中。该论文已被 ACRA 2026 接收作口头报告。
正文 · 原文
Abstract:Deep Interactive Reinforcement Learning (Deep IntRL) accelerates reinforcement learning training by incorporating external advice. However, conventional Deep IntRL uses each piece of advice only in the current step and discards it after a single interaction. Persistent Advising and Broad-Persistent Advising (BPA) extend this process by storing advice in persistent memory and reusing it in similar states through state generalisation based on K-means clustering. This research proposes Encoder BPA, which employs a latent encoder to retrieve and reuse advice. In this framework, a trained encoder is used to map a state sequence into a latent representation at every time step. Advice can be retrieved through latent comparison, together with a top-(k) aggregation and a consistency condition to improve reuse stability. Experimental results show that Encoder BPA improves learning efficiency and advice utilisation at an early stage of training compared with different Deep IntRL approaches, especially in complex and partially observable environments.
| Comments: | 10 pages, 5 figures, 3 tables, Accepted for oral presentation at the Australasian Conference on Robotics and Automation (ACRA 2026) |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13712 [cs.AI] |
| (or arXiv:2611.13712v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13712 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhuoran Liang [view email]
[v1]
Tue, 10 Nov 2026 08:06:06 UTC (1,104 KB)
来源：arXiv cs.AI · arxiv.org