XPM-WM：利用世界模型高效生成多样化协作智能体
Efficient Generation of Diverse Cooperative Agents with World Models
AI 导读
研究者提出 XPM-WM 框架，用学习的世界模型生成模拟轨迹，加速零样本协调（ZSC）中 XPM 等基于种群的训练方法。该方法免去采样 Mixed-Play 轨迹仍能抑制 self-sabotage，并用世界模型引导种群中后续智能体的训练，样本效率显著优于其他 XPM 方法，且可随种群规模线性扩展，生成伙伴的多样性表现与以往方法相当。该训练范式还可推广到 XPM 之外的其他 PBT 方法。
正文 · 原文
Abstract:A major bottleneck in the training process for Zero-Shot Coordination (ZSC) agents is the generation of diverse partner agents via Population-based Training (PBT). Prominent PBT methods, such as Cross-play Minimization (XPM), can be very computationally expensive and sample inefficient, as the training objective requires sampling multiple types of trajectories, in particular the need of Mixed-Play (MP) trajectories to mitigate self-sabotage in cross-play. Each partner agent in the population is also trained from scratch, despite all of the partners in the population learning policies of the same coordination task. In this work, we propose that simulated trajectories from the dynamics model of an environment can drastically speed up the training process for XPM methods. We introduce XPM-WM, a framework for generating simulated trajectories for XPM via a learned World Model (WM). We show XPM with simulated trajectories removes the need to sample Mixed-Play trajectories while still mitigating any self-sabotage. The learned WM also bootstraps the training of subsequent agents in the population. We show that these two factors combined gives XPM-WM significantly better sample efficiency as compared to other XPM methods and scales linearly to the size of the population. We show our proposed method can effectively generate partners with diverse conventions that match the performance of previous methods. Finally, we show that our proposed training paradigm of using learned world models to generate diverse population can be extended to other PBT methods beyond XPM.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2506.07450 [cs.AI] |
| (or arXiv:2506.07450v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2506.07450 arXiv-issued DOI via DataCite |
Submission history
From: Yi Loo [view email]
[v1]
Mon, 9 Jun 2025 05:52:45 UTC (497 KB)
[v2]
Mon, 26 Oct 2026 07:33:58 UTC (506 KB)
来源：arXiv cs.AI · arxiv.org