arXiv cs.AI· Ilia Karpov, Andrey Atamanyuk, Dmitry Ilvovsky, Aleksandr Anokhin, Alex Elizov, Dmitry Zagorulkin, Sergei O. Kuznetsov·· 1 小时前AI 评分39
用结构化表示建模 LLM 推理动态
Modeling LLM Reasoning Dynamics with Structural Representations
AI 导读
一项 arXiv 研究提出 T/E/S 框架,用文本、局部事件与活动区间来刻画语言模型推理的时间结构。
正文 · 原文
Abstract:Language-model reasoning alternates between sustained activities and local operations. We study how this temporal structure predicts continuation and the computation needed to analyze it. T/E/S combines text, local events, and spans of activity. Across reasoning and information-seeking tasks, events and spans improve next-outcome prediction in 70 of 76 evaluation groups for a Transformer and 74 of 76 for Qwen with LoRA. Span duration and preceding events are associated with transitions, and retaining event history across span boundaries improves prediction. We formulate budget-allocation counting and action-conditional continuation queries, prove counting hardness under explicit encodings, and derive conditions for affordable decision certification. In finite-process experiments, bounds that cancel shared continuation mass recover the same optimal actions as full dynamic programming while reducing mean inference CPU time by 77.7%. On 71 held-out HotpotQA and MuSiQue questions, a compact structural observer improves prediction of whether a fixed inference procedure finishes within a CPU allowance: Brier loss falls from 0.0836 to 0.0756. An annotation audit identifies substantial errors in that observer, separating this computational result from semantic validation of its features.
| Comments: | 8 pages, ACM Web Conference (WWW '27) format |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23749 [cs.AI] |
| (or arXiv:2611.23749v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23749 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ilia Karpov [view email]
[v1]
Wed, 18 Nov 2026 03:02:29 UTC (57 KB)
来源:arXiv cs.AI · arxiv.org