arXiv cs.AI· Remy Ogasawara, Cullen Anderson, Narmeen Fatimah Oozeer, Foad Namjoo, Amir Abdullah, Jens Kristian Refsgaard Schou, Jeff M. Phillips·· 4 小时前AI 评分38
LLM 多属性激活转向的正交化与参数化方法
Orthogonal and Parametrized Multi-Attribute Steering of LLMs
AI 导读
arXiv 论文提出参数化转向方法，改进 LLM 推理时的激活转向：不再沿方向推动激活，而是将激活设定在目标特质强度上，使转向强度具有可解释含义。该方法先识别每个转向向量最可转向的区间并重新缩放转向范围；针对多特质组合时向量相互干扰的问题，进一步提出正交化解耦方法，使多个特质可同时转向、互不诱发。
正文 · 原文
Abstract:Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply steering such that the signal is consistently strong and the steering strength has a universally interpretable meaning. We address this first by identifying the most steerable interval for each steering vector and rescaling the steering range accordingly. Building on this, we introduce a more interpretable method for steering, called parameterized steering which instead of pushing activations in a direction, it sets them at the desired trait strength. Next, with multi-trait steering, we find that naively combining vectors causes them to interfere with one another. In response, we develop a method to orthogonalize the vectors; it decouples these traits, allowing a model to exhibit one behavior without inducing the others, thus enabling multiple traits to be steered simultaneously.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37517 [cs.LG] |
| (or arXiv:2610.37517v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37517 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Remy Ogasawara [view email]
[v1]
Tue, 27 Oct 2026 16:39:53 UTC (1,326 KB)
来源：arXiv cs.AI · arxiv.org