SMT：通过模拟审核轨迹越狱函数调用 LLM 的黑盒攻击框架
Beyond the Prompt: Jailbreaking Function-Calling LLMs via Simulated Moderation Traces
AI 导读
研究者提出 SMT，一种针对函数调用 LLM 的黑盒越狱攻击框架，利用多轮模拟审核轨迹把对抗意图分散到执行路径中。在来自五个提供商的商业模型和两个安全基准上，SMT 取得最高的平均攻击成功率和 HarmScore，且查询次数接近最少，代码已开源，论文被 2027 IEEE S&P 接收。
推荐理由
论文揭示了提示词防御覆盖不到的函数调用环境漏洞，并给出多轮模拟审核轨迹攻击框架，对理解工具型 LLM 安全边界有直接参考价值。
正文 · 原文
Abstract:Jailbreak attacks remain a critical threat to the safe deployment of large language models (LLMs). While prior work has primarily studied attacks and defenses at the prompt level, we show that this prompt-centric paradigm overlooks a structural vulnerability in stateful, function-calling environments. In such applications, developer-defined schemas, structured arguments, and untrusted tool outputs are interleaved into a single shared model context. This architecture expands the attack surface by blurring the boundary between trusted control logic and untrusted data, allowing adversarial intent to be distributed across a multi-turn execution path. We exploit this architectural flaw through SMT, a black-box attack framework based on Simulated Moderation Traces. Departing from purely prompt-based interactions, SMT constructs a multi-turn trajectory that simulates a legitimate moderation-auditing workflow. Within this trajectory, a fabricated moderation frame leverages red-team testing as a pretext to elicit harmful generations. The subsequent validation feedback treats safety refusals as execution failures, prompting refinements that gradually weaken the model's safety constraints and ultimately trigger harmful outputs. Extensive empirical evaluations on prominent commercial LLMs from five different providers across two standardized safety benchmarks show that SMT consistently achieves the highest average attack success rate and HarmScore while requiring a near-minimal number of queries, substantially outperforming existing baselines. These findings demonstrate that prompt-level sanitization alone is fundamentally insufficient for defending tool-enabled LLM systems and highlight the urgent need for context-aware validation across schemas, arguments, tool outputs, and accumulated conversation state. The code is available at this https URL.
| Comments: | Accepted to the 2027 IEEE Symposium on Security and Privacy (S&P) |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.00481 [cs.CR] |
| (or arXiv:2607.00481v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2607.00481 arXiv-issued DOI via DataCite |
Submission history
From: Haobo Wang [view email]
[v1]
Wed, 1 Jul 2026 06:08:07 UTC (1,150 KB)
[v2]
Wed, 4 Nov 2026 05:40:42 UTC (1,639 KB)
来源：arXiv cs.AI · arxiv.org