跳到正文
arXiv cs.AI· Sabrina Tang, Nikolai Vo, Carson Murillo, Chira Sukumarnpiti, Sarah Wilson·· 4 小时前AI 评分33

预防性人格向量调控的安全副作用:一项实证审计

Safety Side Effects of Preventative Persona Steering: An Empirical Audit

AI 导读

一项 NeurIPS 2026 Bialign Workshop 收录的实证审计对三个模型上谄媚(sycophancy)与幻觉向量预防性人格向量调控的安全影响进行评估,除特性表达、连贯性和 MMLU 外还测量三项安全基准。结果显示该方法仍会改变拒绝行为,改变幅度取决于目标特性,尚不能证明预防性人格向量调控是安全中立的,应使用多个安全基准评估调控方法。

正文 · 原文

View PDF HTML (experimental)

Abstract:High-level behaviors such as sycophancy and hallucination correspond to directions in a model's activation space (Chen et al., 2025). These directions, known as persona vectors, can suppress undesirable traits either at inference time, subtracting the vector during generation, or through preventative steering, adding the vector during fine-tuning, so the optimizer never needs to encode the trait in weights, then removing the vector at evaluation. Chen et al. (2025) recommend preventative steering because it preserves Massive Multitask Language Understanding (MMLU) accuracy, but validations of this recommendation have measured capability alone. We primarily evaluate results on preventative steering of sycophancy and hallucination vectors on three models, measuring three safety benchmarks alongside trait expression, coherence, and MMLU. Our findings show that preventative steering still changes refusal behavior, but by an amount that depends on the trait and does not establish that preventative persona steering is safety-neutral, and steering methods should be evaluated on several safety benchmarks.
Comments: 5 pages, 1 figure (6 charts), 1 table, accepted at NeurIPS 2026 Bialign Workshop
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.17620 [cs.AI]
  (or arXiv:2611.17620v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17620

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Sabrina Tang [view email]
[v1] Fri, 13 Nov 2026 00:45:18 UTC (1,056 KB)

来源:arXiv cs.AI · arxiv.org