Instrumental Choices：测量 LLM 智能体追求工具性行为倾向的基准
Instrumental Choices: Measuring the Propensity of LLM Agents to Pursue Instrumental Behaviors
AI 导读
一项 arXiv 研究提出基准，测量终端智能体中模型追求工具性收敛（IC）行为（如自我保存）的倾向。评测涵盖十个模型、1,680 个样本，调整后 IC 率为 86/1,680（5.1%），其中两个 Gemini 模型占 66.3% 的 IC 案例，三个任务占 84.9%；IC 行为不可或缺的条件使 IC 率上升最多（+15.7 个百分点）。
正文 · 原文
Abstract:AI systems have become increasingly capable of dangerous behaviours in many domains. This raises the question: Do models sometimes choose to violate human instructions in order to perform behaviour that is more useful for certain goals? We introduce a benchmark for measuring model propensity for instrumental convergence (IC) behaviour in terminal-based agents. This is behaviour such as self-preservation that has been hypothesised to play a key role in risks from highly capable AI agents. Our benchmark is realistic and low-stakes, aiming to reduce evaluation-awareness and roleplay confounds. The suite contains seven operational tasks, each with an official workflow and a shortcut whose policy status varies by condition. An eight-variant shared framework varies monitoring, instruction clarity, stakes, permission, instrumental usefulness and blocked honest paths to support inferences regarding the factors driving IC behaviour. We evaluated ten models using deterministic environment-state scorers over 1,680 samples, with trace review employed for audit and adjudication purposes. The adjusted IC rate is 86 out of 1,680 samples (5.1%). IC behaviour is concentrated rather than uniform: two Gemini models account for 66.3% of IC cases and three tasks account for 84.9%. Conditions in which IC behaviour is indispensable for task success result in the greatest increase in the adjusted IC rate (+15.7 percentage points), whereas emphasising that task success is critical or certain framing choices do not produce comparable effects. Our findings indicate that realistic, low-nudge environments elicit IC behaviour rarely but systematically in most tested models. We conclude that it is feasible to robustly measure tendencies for dangerous behaviour in current frontier AI agents.
| Comments: | 25 pages, 10 figures. Conference submission checklist removed; scientific content unchanged |
| Subjects: | Artificial Intelligence (cs.AI); Computers and Society (cs.CY) |
| Cite as: | arXiv:2605.06490 [cs.AI] |
| (or arXiv:2605.06490v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.06490 arXiv-issued DOI via DataCite |
Submission history
From: Jonas Wiedermann-Möller [view email]
[v1]
Thu, 7 May 2026 16:12:36 UTC (1,902 KB)
[v2]
Fri, 6 Nov 2026 22:03:34 UTC (282 KB)
[v3]
Tue, 10 Nov 2026 17:59:15 UTC (279 KB)
来源：arXiv cs.AI · arxiv.org