LLM 生成时 token 在词表网络上的“舞动”:一项可解释性研究
Dancing Tokens on the Vocabulary Network from LLM Generation
AI 导读
arXiv 论文将六个开源大语言模型(LLM)的输入 embedding 构建成相似性网络,把生成过程映射到网络上分析其几何结构。研究发现该 token-embedding 网络是无标度且重尾分布,但比同协方差的随机 embedding 碎片化得多;蒸馏因 input embedding 大多被继承而几乎不改变拓扑。
正文 · 原文
Abstract:Large language models (LLMs) generate text one token at a time, yet how a particular generation relates to the geometry learned in pre-training remains unclear. The fundamental explainability of LLMs is elusive. Here, we build a similarity network from the input embeddings of six open LLMs and map generation onto it. The token-embedding network is scale-free while heavy-tailed, as are random embeddings with matched covariance, but far more fragmented. Distillation barely changes this topology because input embeddings are largely inherited. The knowledge space is thus deterministic from pre-training. Consecutive generated tokens are rarely graph neighbours and, in most models, no more often than in shuffled order, so generation is not a walk on the predefined network, although the tokens it visits are concentrated within it. Through the logit lens in a layer- wise manner, tokens settle late in depth in most models, and repetitive models remain confined to small domain-specific sub-graphs. These observables could serve as inference-time monitors of degeneration. We thus provide a fresh perspective on understanding LLM generation in the microscopic way from node or link prediction by graph learning.
| Comments: | 24 pages, 6 figures |
| Subjects: | Artificial Intelligence (cs.AI); Disordered Systems and Neural Networks (cond-mat.dis-nn) |
| Cite as: | arXiv:2611.18248 [cs.AI] |
| (or arXiv:2611.18248v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.18248 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Y.-C. Hu [view email]
[v1]
Fri, 13 Nov 2026 14:43:55 UTC (3,328 KB)
来源:arXiv cs.AI · arxiv.org