arXiv cs.AI· Celeste De Schamphelaere, Juan J. Vazquez, Ari Spiesberger, Tom\'a\v{s} Gaven\v{c}iak, Gavin Leech, Euan Ong, Jiahai Feng·· 4 小时前AI 评分50
MAEM 通过扩展激活反演解释语言模型 residual stream 任意方向
Towards interpreting any residual stream direction by scaling activation inversion
AI 导读
研究者提出 Maximally-Activating Example Metamodels（MAEM），用免标签 RL 目标将 residual stream 方向映射为高激活文本示例，突破语料搜索的覆盖限制。
正文 · 原文
Abstract:To interpret a direction in a language model's residual stream, we can look for text that strongly activates it. But when such text is found by scanning a corpus, it is limited to what that corpus contains. We develop Maximally-Activating Example Metamodels (MAEMs), language models which map residual stream directions from a subject model to text exemplars that strongly activate those directions. Our objective is to maximise the cosine similarity of a supplied residual stream direction with the activations of the text generated by the MAEM. This label-free RL objective helps to scale MAEMs on diverse sources of residual stream directions. Empirically, we find the resultant metamodel produces informative exemplars: a Qwen3.6-27B MAEM inverts held-out activations better than corpus search or a natural-language autoencoder (NLA), beats corpus search (at 10M tokens) on a majority of held-out languages/domains, and activates 89% of held-out SAE features. It also identifies concepts behind steering vectors about as well as NLAs, recovers the payloads of LoRA backdoors more often than corpus search, and can even verbalise some model workspace inner monologue. Unlike explanations produced by other metamodels (e.g. NLAs), MAEM exemplars can be verified with a forward pass on the subject model, making the MAEM objective a grounded alternative for scaling the training of metamodels.
| Comments: | Euan Ong and Jiahai Feng advised equally |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.08465 [cs.LG] |
| (or arXiv:2611.08465v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08465 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Juan J Vazquez [view email]
[v1]
Thu, 5 Nov 2026 18:54:40 UTC (428 KB)
来源：arXiv cs.AI · arxiv.org