arXiv cs.AI· Saswat Das, Parvati Viswanathan, Daniel Donnelly, Chang Huang, Sahar Abdelnabi, Ferdinando Fioretto·· 4 小时前AI 评分50
SEABench：衡量自进化智能体内生失调的新基准
SEABench: Benchmarking Endogenous Misalignment In Self-Evolving Agents
AI 导读
研究者发布 SEABench 基准，研究自进化 LLM 智能体因自我修改而产生的内生失调，包含 48 条纵向任务序列，覆盖多种进化面、任务域和伤害类型，并配自适应轨迹发现管线与非进化对照智能体做因果归因。评测显示自进化确实提升任务完成率，但常以安全失败为代价，且安全行为差异体现在链式思维推理中，据此提出的监控策略能以低误报率缓解不安全行为。
正文 · 原文
Abstract:Self-evolving LLM agents have gained prominence for their ability to improve after deployment by modifying their harness, including their controller instructions, memory management protocols, and reusable tools and skills, in response to user and environment feedback. However, locally useful updates may persist into later tasks where they produce unsafe behavior, even without direct adversarial influence. To study this risk, we introduce SEABench, a benchmark for studying endogenous misalignment arising from agent self-evolution, with 48 longitudinal task sequences that span multiple evolution surfaces, task domains, and harm types in a rich personal-assistant environment. To account for the stochasticity inherent in agentic operations, we provide an adaptive trajectory discovery pipeline that probes for failures while preserving original task intent and supports causal attribution through paired non-evolving agents and attribution scores. Our evaluation across multiple recent LLMs, evolution surfaces, and harm types reveals that self-evolution indeed increases task completion rates but often at the cost of safety failures that are absent for paired non-evolving baseline agents. We also show that qualitatively different safety behaviors emerge across evolution surfaces and harm types. Further, we show that this divergence in safety behavior is reflected in agents' chain-of-thought reasoning, which yields an effective monitoring strategy that can mitigate unsafe behavior with a low false positive rate.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2609.35596 [cs.CR] |
| (or arXiv:2609.35596v3 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2609.35596 arXiv-issued DOI via DataCite |
Submission history
From: Saswat Das [view email]
[v1]
Mon, 28 Sep 2026 16:46:25 UTC (1,531 KB)
[v2]
Tue, 29 Sep 2026 20:10:34 UTC (1,531 KB)
[v3]
Wed, 4 Nov 2026 21:54:41 UTC (1,528 KB)
来源：arXiv cs.AI · arxiv.org