arXiv 论文提出评估 LLM 防护部署安全的框架,分析 52 篇论文 85 项评测的证据缺口
The Safeguard Worked. Is the LLM System Safer?
AI 导读
论文提出一个框架,将已发表的防护评测结果换算为指定结果与威胁模型下攻击者仍可获得的有害协助的最小与最大值,并应用于 52 篇论文的 85 项防护评测,其中仅 5 项论证所有攻击路径经过检查点、4 项提供拦截后残余协助的证据。
正文 · 原文
Abstract:Safeguards for large language model services are typically evaluated by their attack-blocking rates on test suites. Deployment decisions also depend on how much harmful assistance attackers can still obtain from the system, which these rates do not measure. To determine what published evaluations establish about deployment safety, we develop a framework that translates reported results into the minimum and maximum harmful assistance that can remain under a specified outcome and threat model. For safeguards that block attacks through a check, we show that the most an attacker can obtain depends on three properties: whether attack paths pass through the check, its miss rate against every attacker in the threat model, and how much assistance remains after a block. We apply this framework to 85 safeguard evaluations from 52 papers. Of these, only five argue that every attack path passes through the check, and four provide evidence on what remains after a block. These evidence gaps have practical consequences: in NeMo Guardrails' default streaming configuration, the check blocked 94% of password leaks in our test, yet the client received the password in every leak, blocked or not. We identify which of the three properties each safeguard category's evaluations omit, and what to measure for each of four ways attacks bypass a check. Deployments that require these measurements before launch can choose and configure safeguards that limit what attackers obtain from the deployed system.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.00519 [cs.CR] |
| (or arXiv:2609.00519v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2609.00519 arXiv-issued DOI via DataCite |
Submission history
From: Pingyu Wu [view email]
[v1]
Tue, 1 Sep 2026 00:38:35 UTC (161 KB)
[v2]
Tue, 17 Nov 2026 14:26:46 UTC (101 KB)
来源:arXiv cs.AI · arxiv.org