arXiv cs.AI· Weijiang Lv, Wentong Zhao, Jiayu Wang, Yuhao Wu, Jiaheng Wei, Xiaobo Xia·· 4 小时前AI 评分39
GeoFaith：基于时空潜在动力学为 LLM 学习忠实推理过程
Learning Faithful Reasoning Processes with Spatio-Temporal Latent Dynamics for Large Language Models
AI 导读
GeoFaith 是一个无需过程奖励模型的时空潜在动力学框架，用于学习大语言模型的忠实推理过程。它将终态潜在不确定性与预测熵动力学作为过程正则化融入 GRPO，惩罚过早坍缩、振荡和长期不确定性等异常模式，无需步骤级过程标注即可联合优化答案正确性与推理质量。实验显示 GeoFaith 持续提升推理准确率，生成更高效的推理轨迹，优于仅结果奖励的强化学习和过程监督基线。
正文 · 原文
Abstract:Chain-of-Thought (CoT) reasoning has significantly enhanced the reasoning capabilities of large language models (LLMs). However, existing reinforcement learning methods primarily optimize outcome-level rewards, providing limited supervision over the reasoning process. As a result, models may produce shortcut-prone or unstable reasoning trajectories even when the final answer is correct. We propose GeoFaith, a spatio-temporal latent dynamics framework for learning faithful reasoning processes without process reward models. GeoFaith combines final-state latent uncertainty, which measures whether a completed trajectory lies in a well-supported manifold region, with predictive-entropy dynamics, which penalize abnormal uncertainty patterns such as premature collapse, oscillation, and prolonged uncertainty. These signals are incorporated as process regularization into GRPO, jointly optimizing answer correctness and reasoning quality without step-level process annotations. Experiments show that GeoFaith consistently improves reasoning accuracy, produces more efficient reasoning trajectories, and outperforms both outcome-only reinforcement learning and process-supervision baselines.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.26893 [cs.CL] |
| (or arXiv:2605.26893v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26893 arXiv-issued DOI via DataCite |
Submission history
From: Xiaobo Xia [view email]
[v1]
Tue, 26 May 2026 11:54:40 UTC (10,648 KB)
[v2]
Mon, 26 Oct 2026 12:02:16 UTC (6,864 KB)
来源：arXiv cs.AI · arxiv.org