将授权委托给未对齐智能体：联盟对齐与安全控制
Delegating Authorization to Misaligned Agents: Coalitional Alignment and Safe Control
AI 导读
一篇 arXiv 论文提出“k-鲁棒联盟对齐”概念，解决将 AI 智能体行为审查委托给其他未对齐智能体时的安全控制问题。论文证明，容忍 k 个否决的阈值规则安全当且仅当剩余审查者的效用可写成主委托方效用的非负组合，并将该刻画推广到折扣 MDP 的序贯控制与策略性投票场景。对现有审查模型的实验表明，即使没有个体对齐的审查者并容忍部分否决，集体审查仍可保持安全。
正文 · 原文
Abstract:Long-running AI agents create a control problem: each action they take changes the state, which in turn affects the trajectory of future actions. If the agent is not fully aligned, then guaranteeing safety requires approving consequential actions before allowing them to be executed. But requiring human approval at every step makes attention a bottleneck. Delegating review to other AI agents raises the same alignment problem: the reviewers may themselves be misaligned. We identify a condition on a reviewing panel that is weaker than individual alignment yet necessary and sufficient for a guarantee that the principal fares at least as well in expectation as under a designated baseline policy.
Each reviewer agent reports whether an action proposal made by a proposer agent improves its own utility relative to the baseline. We show that a threshold rule tolerating $k$ disapprovals is safe exactly when, after any $k$ reviewers are removed, the principal's utility can be written as a nonnegative combination of the remaining reviewers' utilities, plus a term that is nonnegative on every feasible proposal. We call this property $k$-robust coalitional alignment. The characterization lifts to sequential control: in a discounted MDP with an arbitrary proposer agent, safety at every state is both necessary and sufficient for the induced policy to match or improve on the baseline.
When reviewers vote strategically, full-panel coverage in reward-function space guarantees that every Nash equilibrium is safe under the unanimous approval rule; in contrast, more permissive thresholds can admit unsafe equilibria even when reviewers are individually aligned.
Experiments with existing reviewer models show that collective review can remain sound without an aligned individual, even when some disapprovals are tolerated.
| Subjects: | Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Theoretical Economics (econ.TH) |
| Cite as: | arXiv:2609.15803 [cs.GT] |
| (or arXiv:2609.15803v2 [cs.GT] for this version) | |
| https://doi.org/10.48550/arXiv.2609.15803 arXiv-issued DOI via DataCite |
Submission history
From: Sikata Sengupta [view email]
[v1]
Mon, 14 Sep 2026 16:12:41 UTC (196 KB)
[v2]
Wed, 11 Nov 2026 21:21:46 UTC (198 KB)
来源：arXiv cs.AI · arxiv.org