arXiv cs.AI· Diancheng Kang, Zheyuan Liu, Ningshan Ma, Yue Huang, Zhaoxuan Tan, Meng Jiang·· 3 小时前AI 评分41
Prompt-Activation Duality：通过注意力层干预改进激活引导
Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions
AI 导读
arXiv 论文提出 Gated Cropped Attention-Delta steering（GCAD），从系统提示词对自注意力的贡献中提取引导信号，并以 token 级门控方式施加，用于解决有状态对话中 KV-cache 污染导致的激活引导失效问题。
正文 · 原文
Abstract:Activation steering controls language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache contamination as a key failure mode: steered token states are stored and repeatedly reused, turning a local perturbation into cumulative coherence degradation. To address this challenge, we propose Gated Cropped Attention-Delta steering (GCAD), which extracts steering signals from system-prompt contributions to self-attention and applies them with token-level gating. Across persona-steering experiments, GCAD preserves trait control while substantially improving long-horizon coherence. On the main multi-turn benchmark, GCAD improves average coherence drift from -18.6 to -1.9 and raises turn-10 trait expression from 78.0 to 93.1. These results suggest that activation steering becomes more reliable when interventions follow the prompt-mediated pathways that models already use for behavioral control.
| Comments: | NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.10664 [cs.CL] |
| (or arXiv:2605.10664v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.10664 arXiv-issued DOI via DataCite |
Submission history
From: Zheyuan Liu [view email]
[v1]
Mon, 11 May 2026 14:44:32 UTC (7,026 KB)
[v2]
Thu, 14 May 2026 04:08:33 UTC (7,026 KB)
[v3]
Tue, 10 Nov 2026 22:35:27 UTC (8,152 KB)
来源：arXiv cs.AI · arxiv.org