MAVIC：多智能体强化学习中的鲁棒指令遵从方法
Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning
AI 导读
研究者提出 Macro-Action Value Correction for Instruction Compliance（MAVIC），用于解决多智能体强化学习中自然语言指令打断宏观动作时 Bellman 更新导致价值估计不一致的问题。
正文 · 原文
Abstract:Multi-agent reinforcement learning (MARL) in real-world use cases may need to adapt to external natural language instructions that interrupt ongoing behavior and conflict with long-horizon objectives. However, conditioning rewards on instructions introduces a fundamental failure mode as Bellman updates couple value estimates across instruction contexts, leading to inconsistent values when instructions interrupt macro-actions. We propose Macro-Action Value Correction for Instruction Compliance (MAVIC), which corrects Bellman backups at instruction boundaries by correcting the incoming instruction objective and restoring the continuation value under the current objective. Unlike reward shaping, MAVIC modifies the bootstrapping target itself, enabling consistent value estimation under stochastic instruction switching within a unified policy. We provide theoretical analysis and an actor-critic implementation, and show that MAVIC achieves high instruction compliance while preserving base task performance in increasingly complex cooperative multi-agent environments.
| Comments: | NeurIPS 2026 Atlanta Poster |
| Subjects: | Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2605.12655 [cs.AI] |
| (or arXiv:2605.12655v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.12655 arXiv-issued DOI via DataCite |
Submission history
From: Wo Wei Lin [view email]
[v1]
Tue, 12 May 2026 19:01:16 UTC (356 KB)
[v2]
Fri, 5 Jun 2026 16:06:40 UTC (2,744 KB)
[v3]
Wed, 10 Jun 2026 15:03:44 UTC (2,744 KB)
[v4]
Thu, 29 Oct 2026 15:51:53 UTC (911 KB)
来源：arXiv cs.AI · arxiv.org