ICA Lens 论文提出以非高斯性直接发现语言模型激活中的可解释方向
ICA Lens: Non-Gaussianity Reveals Interpretable Directions in Language Models
AI 导读
Sida Liu 等人在 arXiv 论文 2606.11722 提出 ICA Lens,发现 SAE 解码器方向的投影分布明显比随机方向更非高斯,可直接在 LLM 残差流激活中搜索非高斯方向来发现可解释特征。
正文 · 原文
Abstract:Finding human-interpretable directions in language-model activations is important for understanding and controlling model behavior. These directions are usually discovered through indirect learning objectives such as sparse reconstruction, but it remains unclear what existing statistical structure distinguishes interpretable directions and whether that structure can be used directly for feature discovery. We find that non-Gaussianity provides such a signature: projection distributions of natural-text residual-stream activations along sparse autoencoder (SAE) decoder directions are substantially more non-Gaussian than those along random directions, with most contexts forming a concentrated background and recurring patterns often occupying one tail. This raises a natural question: if interpretable directions tend to have non-Gaussian projection distributions, can we discover them by searching for non-Gaussian directions directly in activation space, without training another sparse dictionary? To test this hypothesis, we introduce ICA Lens, a practical workflow for efficient discovery, reliable annotation, and faithful evaluation of non-Gaussian directions in LLM residual-stream activations. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find that (1) the recovered non-Gaussian directions capture coherent lexical, syntactic, and semantic patterns under both human and LLM-based evaluation; (2) non-Gaussianity is positively associated with both human-label confidence and automated interpretability; and (3) the resulting coordinates are competitive with SAEs across interpretability, sparse reconstruction, concept prediction, and causal interventions, while fitting in minutes per layer and using 7-43 times less memory. These results show that non-Gaussianity is a direct unsupervised signal for discovering interpretable and useful structure in activations.
| Comments: | Added experiments and refined the presentation |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2606.11722 [cs.LG] |
| (or arXiv:2606.11722v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2606.11722 arXiv-issued DOI via DataCite |
Submission history
From: Feijiang Han [view email]
[v1]
Wed, 10 Jun 2026 06:53:58 UTC (2,296 KB)
[v2]
Mon, 19 Oct 2026 17:23:26 UTC (6,395 KB)
[v3]
Tue, 17 Nov 2026 04:23:38 UTC (7,693 KB)
来源:arXiv cs.AI · arxiv.org