中科大团队论文揭示 LLM 滥用防护普遍依赖不可信输入的高风险漏洞
LLM Misuse Safeguards Rely on Untrusted Input: A Widespread, High-Risk Vulnerability
AI 导读
Pingyu Wu、Weiming Zhang、Nenghai Yu 的论文发现 LLM 滥用防护普遍依赖用户书写内容判断是否放行答案,攻击者伪造专业身份即可绕过。
推荐理由
论文指出滥用防护依赖不可信的用户输入,并给出覆盖 129 篇顶会论文的证据与补丁思路,为安全研究提供反思视角。
正文 · 原文
Abstract:Large language models now help professionals in fields such as cybersecurity and biology, yet the same answers would help an attacker. Deployers rely on misuse safeguards to keep such answers from attackers. We find that these safeguards share a high-risk vulnerability: they decide whether to release an answer from what users write, including the professional identity or purpose they claim, and anyone can write such a claim. Threat reports from model vendors and security firms show that attackers who fabricate a professional identity or purpose already exploit this vulnerability to cause serious harm. The vulnerability is also widespread across the state of the art: every defense proposed in 129 papers from the latest editions of 12 top venues has it. Safeguard research studies the wrong object: it treats harm as a property of the text of a request, although the same text can come from a professional or an attacker. Current safeguard benchmarks therefore cannot reveal the vulnerability: a safeguard can score high on them and still give dangerous answers to attackers. Because the vulnerability lies in what users write, no improvement in how safeguards judge that writing can close it. Under the patch we propose, large language models can help professionals without giving dangerous capabilities to anyone who merely claims to be one.
| Comments: | 13 pages, 4 tables. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.27951 [cs.CR] |
| (or arXiv:2607.27951v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2607.27951 arXiv-issued DOI via DataCite |
Submission history
From: Pingyu Wu [view email]
[v1]
Thu, 30 Jul 2026 09:59:08 UTC (249 KB)
[v2]
Tue, 17 Nov 2026 16:34:38 UTC (124 KB)
来源:arXiv cs.AI · arxiv.org