arXiv 论文研究工具使用型 AI Agent 的弃权操纵攻击：Gemini 过度弃空率从 9.8% 升至 86.3%
When Safety Becomes an Attack Surface: Manipulating Abstention in Tool-Using AI Agents
AI 导读
arXiv 论文（2611.14864）研究警告诱导式过度弃权攻击，攻击者在低信任内容中植入伪造风险警告，阻止 AI Agent 完成合法任务。
正文 · 原文
Abstract:An AI agent's decision not to act can itself become an attack surface. We study warning-induced over-abstention, an availability attack in which adversaries place fabricated risk warnings in low-trust content to prevent legitimate task completion. Our central premise is that safe abstention requires evaluating both the claimed risk and the authority of the evidence supporting it. We develop controlled counterfactual task variants that vary risk content, source provenance, and warning wording, and evaluate a Verify-Before-Abstain protocol that corroborates warnings against authoritative records or the underlying content. Across three model families and 51 injectable AgentAbstain task pairs, caution-first prompting amplifies the attack. For Gemini, induced over-abstention rises from $9.8\%$ to $86.3\%$, despite unchanged clean-task completion. Matched wording comparisons further reveal sensitivity to warning language, with substantial variation across models and environments. Provenance-aware verification improves legitimate completion, while forgery and sensitive-content controls expose its dependence on authoritative evidence and its limitations in preserving confidentiality. These results motivate evaluating agent safety through the justification for restraint, alongside the safety of actions taken.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14864 [cs.CR] |
| (or arXiv:2611.14864v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14864 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Charles Xiao [view email]
[v1]
Wed, 11 Nov 2026 00:36:25 UTC (44 KB)
来源：arXiv cs.AI · arxiv.org