跳到正文
arXiv cs.AI· Keshav Shenoy, Anthony Zang, Eric Gan, Jacqueline Tay, Amy Ngo, Sebastian Prasanna, Julian Stastny, Vivek Hebbar·· 56 分钟前精选AI 评分63

arXiv 论文:审计者难以从 AI 安全研究端到端记录中识别破坏行为

Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research

AI 导读

arXiv 论文(arXiv:2611.21832)通过红蓝对抗实验研究 AI 安全研究中的破坏检测,由人类专家充当破坏者开展实验,审计者判断哪些被破坏及如何破坏。

推荐理由

论文用红蓝对抗实验量化了当前审计方法识别研究破坏的能力,给出了人类与前沿模型的具体检出数据,可用于安全评测设计参考。

正文 · 原文

View PDF HTML (experimental)

Abstract:As AI systems are increasingly used to automate AI R&D, misaligned systems could sabotage research intended to improve model safety and alignment. To stress-test defenses against such sabotage, we use a human expert as a proxy saboteur on AI safety research. We conduct a red-team/blue-team game where a human researcher runs AI safety experiments from scratch and auditors try to tell which were sabotaged and how. Given access to the full research transcript and the ability to run experiments, human auditors mostly fail to distinguish sabotaged from benign research: they achieve an AUROC of 0.59, not significantly better than chance, and fully identify the specific sabotage in only 17% of cases. Despite not being allowed to run experiments, the best frontier models outperform human auditors (Fable 5.1: AUROC 0.73, 32% full identification) within noise and weaker models perform comparably. When the saboteur chooses which experiments are attacked, rather than being required to sabotage every question given, sabotage identification drops further. Our results suggest that research sabotage by expert-level agents is difficult to detect with current auditing methods and provide a new setting for studying research sabotage.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21832 [cs.AI]
  (or arXiv:2611.21832v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21832

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Keshav Shenoy [view email]
[v1] Mon, 16 Nov 2026 19:52:38 UTC (669 KB)

来源:arXiv cs.AI · arxiv.org