跳到正文
arXiv cs.AI· Peng Yu, Xiaoyu Wen, Zhida He, Ziyuan Zhou, Han Qi, Shao Zhang, Qiaosheng Zhang, Ying Wen, Chaochao Lu·· 56 分钟前AI 评分28

DACE:多样性驱动的对抗协同演化框架,提升 LLM 安全对齐鲁棒性

DACE: Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment

AI 导读

arXiv 论文提出 DACE,一种多样性驱动的对抗协同演化框架,针对 LLM 安全对齐中的攻击策略坍缩和防御对抗遗忘两类问题。攻击端引入风险类别×攻击风格二维策略空间和归一化边际覆盖增益奖励,防御端维护基于 Beta–Bernoulli 后验与 Thompson 采样的贝叶斯对抗回放池。在标准安全、自动化攻击与通用能力 benchmark 上,DACE 提升对分布外攻击的鲁棒性并保留通用能力。

正文 · 原文

View PDF HTML (experimental)

Abstract:Ensuring robust safety alignment of large language models (LLMs) is increasingly difficult as adversarial attacks evolve and outpace alignment pipelines built on pre-collected data. Existing co-evolutionary frameworks let the defender chase a moving attacker, yet their training dynamics expose two compounding pathologies: \emph{attack strategy collapse}, where the attacker overfits to a narrow set of high-reward rewrites, and \emph{defense adversarial forgetting}, where the defender loses competence against earlier attacks as the attack distribution drifts. Existing remedies are partial: attacker-side diversity rewards score textual novelty rather than strategy novelty, and defender-side replay relies on discrete judge scores that ignore estimation uncertainty, response stochasticity, and defender non-stationarity. We introduce \textbf{\method}, a diversity-driven adversarial co-evolution framework that targets both pathologies jointly. On the attacker side, \method couples an explicit two-dimensional strategy space (risk category $\times$ attack style) with a \emph{normalized marginal coverage gain} reward, providing a bounded, non-vanishing exploration signal at the strategy layer. On the defender side, \method maintains a unified \emph{Bayesian adversarial replay pool} whose Beta--Bernoulli threat posteriors are refreshed by time decay and sampled via Thompson sampling, anchoring defender training to an evolving threat landscape. Across standard safety, automated-attacker, and general-capability benchmarks, \method improves robustness to out-of-distribution attacks while preserving general capabilities, and yields broader attacker strategy coverage.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22278 [cs.AI]
  (or arXiv:2611.22278v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.22278

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Peng Yu [view email]
[v1] Tue, 17 Nov 2026 06:27:03 UTC (466 KB)

来源:arXiv cs.AI · arxiv.org