arXiv cs.AI· Andrew McAlinden, Caroline Wang, Steven Shi, Arrasy Rahman, Elad Liebman, Peter Stone·· 2 小时前AI 评分34
N智能体临时协作中的集中式 Critic 研究
Centralized Critics for N-Agent Ad Hoc Teamwork
AI 导读
arXiv 论文研究 N-agent ad hoc teamwork（NAHT）中集中式 critic 的设计，比较两类优势估计器：仅依赖己方智能体观测与动作历史的 controlled history 估计器，以及额外纳入不可控队友历史的 joint history 估计器。
正文 · 原文
Abstract:Centralized training with decentralized execution (CTDE) has become the dominant paradigm in multi-agent reinforcement learning (MARL), motivating extensive study of what privileged information should be made available to the critic during training. While prior work has examined the role of global state in fully cooperative settings, the design of centralized critics for N-agent ad hoc teamwork - in which dynamically varying set of learning agents must coordinate with an arbitrary set of uncontrolled teammates drawn from an unknown distribution - remains poorly understood. We study this question by comparing two natural classes of advantage estimators: controlled history estimators, which condition only on the observation and action histories of the learner's own agents, and joint history estimators, which additionally condition on the histories of the uncontrolled teammates. We prove that both estimators are unbiased and characterize the conditions under which the joint history estimate attains strictly lower variance than its controlled counterpart. Empirically, joint history critics match or outperform controlled history critics across a range of NAHT tasks, yielding improved sample efficiency and final returns. Together, these results provide both a theoretical justification and an empirical recommendation for incorporating uncontrolled agent histories into centralized critics for N-agent ad hoc teamwork.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.34799 [cs.AI] |
| (or arXiv:2610.34799v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34799 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Andrew McAlinden [view email]
[v1]
Mon, 26 Oct 2026 03:41:48 UTC (2,681 KB)
来源：arXiv cs.AI · arxiv.org