神经网络中的表示原子：基本表示单元的定义与评估
Representational Atoms in Neural Networks
AI 导读
arXiv 论文（arXiv:2611.14168）形式化定义了神经网络的基本表示单元（FRUs），并提出保真度与稳定性两项评估标准。研究证明阈值激活稀疏自编码器（TSAEs）在适当条件下能以高保真度和稳定性识别 FRUs，并通过大规模实验验证。稳定性更高的 FRUs 表现出更强的单语义性，可跨层组合恢复对应人类概念结构的语义组织。
正文 · 原文
Abstract:Understanding the internal representations of neural networks and characterizing their relationship with human conceptual structure are two critical questions for elucidating model mechanisms. The former reveals how learned knowledge is organized in internal representations, whereas the latter provides a basis for interpreting these representations through human conceptual structure. Here, we formally define fundamental representational units (FRUs) and establish two evaluation criteria: fidelity and stability. We further prove that threshold-activated sparse autoencoders (TSAEs) can identify FRUs with high fidelity and stability under appropriate conditions, and validate this result through large-scale experiments. Further analyses reveal that FRUs with higher stability exhibit greater monosemanticity and can be composed across layers to recover semantic organization corresponding to human conceptual structure. These results provide a unified framework for defining, evaluating and identifying FRUs in neural networks, and establish a foundation for characterizing the relationship between neural network representations and human conceptual structure.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14168 [cs.CL] |
| (or arXiv:2611.14168v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14168 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chenhui Hu [view email]
[v1]
Tue, 10 Nov 2026 14:56:23 UTC (27,134 KB)
来源：arXiv cs.AI · arxiv.org