通过迭代博弈追踪 LLM 推理链中的内部表示
Tracking Representations in LLM Reasoning Traces for Iterated Games
AI 导读
一项 arXiv 论文(被 SoCalNLP 2026 接收)用机制可解释性技术研究 LLM 智能体在不完全信息迭代博弈中的决策。研究发现模型在 CoT 过程中形成并迭代修正对博弈的内部表示,一旦过早锁定错误信念便很难恢复,这一失败同时体现在推理链文本和层激活中。该工作为诊断多智能体系统的策略推理错误提供方向,并提供了交互式可视化工具。
正文 · 原文
Abstract:Large language model (LLM) agents achieve state-of-the-art performance across domains through chain-of-thought (CoT) reasoning. Despite their ubiquitous use, little is understood about the relationship between the externalized trace and the model's actual computation. We address this by investigating how LLM agents make decisions in iterative games with incomplete information. Using a series of mechanistic interpretability techniques, we find that the model forms and iteratively revises an internal representation of the game throughout the CoT. Once the model prematurely commits to an incorrect belief, it rarely recovers. This failure is manifested in both the literal trace and the layer activations. Ultimately, as multi-agent systems are more prevalent, our work opens the door to a more focused diagnosis of their strategic reasoning errors. For illustrative purposes, we offer an interactive visualizer of our methodology on selected sample traces at this https URL
| Comments: | Accepted to SoCalNLP 2026. 15 pages, 11 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT) |
| Cite as: | arXiv:2611.23403 [cs.CL] |
| (or arXiv:2611.23403v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23403 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chaipat Tirapongprasert [view email]
[v1]
Tue, 17 Nov 2026 19:03:18 UTC (676 KB)
来源:arXiv cs.AI · arxiv.org