arXiv cs.AI· Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, Leo Yu Zhang·· 6 小时前AI 评分51
JevAdvBench：首个类型化决策模型对抗鲁棒性基准与黑盒攻击研究
JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models
AI 导读
论文提出 JevAdvBench，据作者所知是首个针对类型化决策模型（TDM）的统一对抗鲁棒性基准，含 862 道经审核的题目和 12 种黑盒攻击方法。攻击可翻转 46.3% 的 Jev 正确决策，而相同重跑仅翻转 0.17%；无命令的有缺陷论证可翻转 77.2%，接近直接提示词注入的 92.0%。
正文 · 原文
Abstract:Typed decision models (TDMs), such as Jev, answer a question about an input with a probability, an option, or an ordinal level. Web services execute these decisions automatically above a confidence threshold. An attacker who writes part of the input can change a decision that the service then executes. Current adversarial benchmarks cannot measure this risk, because they judge generated text. A TDM writes no text, and its wrong decisions have a valid format. We present JevAdvBench, to our knowledge the first unified benchmark for the adversarial robustness of TDMs. It contains 862 questions with audited answers and twelve black-box attack methods that edit the input or the options by adaptive search. Because a changed decision is not always wrong, a review keeps only 5,586 answer-preserving variants, and we attack only correct decisions. The attacks flip 46.3% of Jev's correct decisions, while identical re-runs flip only 0.17%. A flawed argument without a command flips 77.2%, close to direct prompt injection at 92.0%. The same variants flip 34.0% to 51.7% of correct decisions on eight open TDMs. At a 0.8 threshold, 23.5% of the decisions that the service still executes are wrong. On Agentgateway, the Jev guardrail rejects a jailbreak request but accepts it with an inserted argument. Web services should therefore treat TDM inputs as untrusted and should not act on confidence alone. Project website: this https URL
| Comments: | 12 pages, 5 figures. Project page: this https URL |
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.31142 [cs.CR] |
| (or arXiv:2609.31142v2 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2609.31142 arXiv-issued DOI via DataCite |
Submission history
From: Hangtao Zhang [view email]
[v1]
Fri, 25 Sep 2026 11:32:19 UTC (588 KB)
[v2]
Mon, 26 Oct 2026 15:14:23 UTC (439 KB)
来源：arXiv cs.AI · arxiv.org