arXiv cs.AI· Arnab Sen Sharma, Eric Todd, Sheridan Feucht, Andy Arditi, Rohit Gandikota, Natalie Shapira, Gabriele Sarti, Constanza Fierro, Lisa Schut, David Bau·· 3 小时前AI 评分51
arXiv 论文提出 LLM 目标编码机制：稀疏注意力头定位子目标并提出 NRGS 解释
Goal Encoding in LLMs
AI 导读
arXiv 论文（arXiv:2611.17297）研究 LLM 如何编码目标，在多子任务编程场景中把下一子目标的编码定位于一组稀疏注意力头的输出，同一组头还能用自然语言解码模型未来动作。
正文 · 原文
Abstract:We investigate whether and how LLMs encode goals: internal neural representations that precede and guide action, and that are flexible enough to be realized through many different execution paths. We construct a coding setting with multiple sub-tasks that allows us to evaluate goal-seeking behavior and the selection of sub-goals. In this setting, we localize an encoding of the next sub-goal in the outputs of a sparse set of attention heads. The same heads can be used to decode the LM's future actions in natural language (e.g., "I'm checking if a word is a palindrome"). We verify that this encoding is causal for next action and is plastic: transplanting it into a different environment, e.g., another programming language, triggers the same operation realized with a different sequence of tokens. The LM also sustains the encoding throughout the action's execution, which is puzzling because a transformer LM without recurrence cannot carry a state forward for more tokens than it has layers left. We propose that it does not need to. Under a non-recurrent goal-seeking (NRGS) account, the LM frequently re-reads the goal from the task specification, compares it with the progress visible in the growing context, and pursues what remains. We test the predictions of this account with targeted causal interventions.
| Comments: | Project page, code and data: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.17297 [cs.CL] |
| (or arXiv:2611.17297v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17297 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arnab Sen Sharma [view email]
[v1]
Thu, 12 Nov 2026 18:55:06 UTC (2,065 KB)
来源：arXiv cs.AI · arxiv.org