arXiv 论文提出通过 MoE 路由抑制推理模型过度思考，token 用量中位数降 30.4%
Routes of Reasoning: Reading and Reshaping Test-Time Compute in MoE Models
AI 导读
arXiv 论文（2611.05919）发现 MoE 模型的 router 是免训练的行为控制接口，可将推理轨迹按规划、执行、重述、事实、反思、结论六种行为标注，router logits 上的线性探针用约一半特征即可接近残差流探针的预测效果，且跨六数据集、五领域泛化。
正文 · 原文
Abstract:Reasoning models allocate test-time compute across distinct behaviors such as planning, deriving, and verifying. However, models frequently "overthink," continuing to verify and re-derive long after finding a correct answer. This wastes expensive compute and can even degrade performance. Existing interventions, such as token budgets or steering, primarily regulate reasoning duration rather than targeting specific behaviors. In this work, we demonstrate that Mixture-of-Experts (MoE) routers provide a natural, training-free interface for behavioral control. We first split reasoning traces into sentence-level segments and label each with one of six reasoning behaviors (planning, execution, restatement, fact, reflection, and conclusion), so that every token carries the behavior of its segment. Linear probes on router logits predict this label nearly as well as probes on the residual stream, the model's full hidden state and the standard target for probing, while using about half as many features. These signals generalize: experts associated with a reasoning behavior on one dataset predict the same behavior on the others, across six datasets spanning five domains (math, code, science, knowledge, and commonsense). Building on this, we selectively edit expert router scores during inference to curb overthinking. Across four models and four benchmarks, this intervention reduces the number of generated tokens by a median of 30.4% (and up to 63.3%). Crucially, it encourages the model to stop once the problem is solved, rather than skipping essential reasoning steps. Accuracy never drops by more than 0.5 percentage points, and frequently improves. Ultimately, MoE routing offers both an interpretable lens into reasoning states and a lightweight mechanism for optimizing test-time compute.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.05919 [cs.AI] |
| (or arXiv:2611.05919v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05919 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shengjie Wang [view email]
[v1]
Wed, 4 Nov 2026 04:13:02 UTC (317 KB)
来源：arXiv cs.AI · arxiv.org