跳到正文
arXiv cs.AI· Jianhong Wang, Yang Li, Samuel Kaski, Jonathan Lawry·· 4 小时前AI 评分35

开放多智能体系统强化学习的公理化框架:Shapley Machines 与 Banzhaf Machines(NeurIPS 2026)

An Axiomatic Framework for Reinforcement Learning in Open Multi-Agent Systems

AI 导读

一篇发表于 NeurIPS 2026 的 arXiv 论文提出面向开放多智能体系统的公理化强化学习框架,基于 n-agent ad hoc teamwork(NAHT),从 Shapley 公理(Efficiency、Symmetry、Linearity)出发推导价值函数学习,并证明对个体价值函数施加这些公理可恢复 Dec-POMDP 的 Shapley value。

正文 · 原文

View PDF HTML (experimental)

Abstract:Open multi-agent systems, where only a subset of agents are controllable, are central to real-world domains such as smart grids, yet lack a principled foundation for reinforcement learning algorithm design. We study this setting through n-agent ad hoc teamwork (NAHT), a recent framework for learning in open multi-agent environments. We propose an axiomatic framework that derives value function learning from the Shapley axioms (Efficiency, Symmetry and Linearity), treating them as structural constraints rather than heuristic design choices. This formulation builds on a novel NAHT decomposition, which characterises the fundamental difference between open and fully controllable multi-agent systems, and establishes a cooperative game-theoretic foundation for learning. We further show that enforcing these axioms on learning individual value functions recovers the Shapley value for Dec-POMDPs, providing a principled bridge between reinforcement learning and classical cooperative game theory. Building on this framework, we introduce Shapley Machines and Banzhaf Machines, two generic algorithmic templates that can be instantiated with standard reinforcement learning methods. When applied to strong base algorithms such as IPPO and POAM, our methods consistently improve learning efficiency and generalisation to unseen agent types and conventions across NAHT benchmark tasks. The code of experiments is published on this https URL.
Comments: 43 pages, Published at NeurIPS 2026
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA)
Cite as: arXiv:2611.03301 [cs.LG]
  (or arXiv:2611.03301v3 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.03301

arXiv-issued DOI via DataCite

Submission history

From: Jianhong Wang [view email]
[v1] Mon, 2 Nov 2026 15:36:22 UTC (1,646 KB)
[v2] Tue, 3 Nov 2026 13:15:29 UTC (1,646 KB)
[v3] Thu, 12 Nov 2026 20:09:27 UTC (1,647 KB)

来源:arXiv cs.AI · arxiv.org