逻辑一致且准确的预测:微调监督模型提升 LLM 预测一致性
Logically Consistent Accurate Forecasts
AI 导读
基于 LLM 的智能体预测系统已能与人类专家比肩,但相关事件预测常不满足概率公理,简单投影强行纠正又会损害准确率。论文微调一个监督语言模型,利用每个预测背后的推理过程同时消除不一致并提升准确率。在 ForecastBench 上,该方法将 GPT-5.1 检索增强预测器的 Brier 分数改进 9.3%,结构性不一致指标降低 82-90%,且基线与消融实验显示无替代方法能同时达到此效果。
正文 · 原文
Abstract:Agentic forecasting systems based on large language models (LLMs) are now competitive with expert human forecasters, but these systems still suffer from a critical limitation: the inability to produce forecasts that are logically consistent. For example, forecasts for related events often fail to satisfy the axioms of probability, which in turn implies that related forecasts cannot simultaneously be correct and undermines their utility for decision makers. Enforcing consistency is not trivial -- naive approaches (e.g., projection onto the set of consistent forecasts) can degrade accuracy. We argue that the key to overcoming the apparent tradeoff between consistency and accuracy lies in the reasoning which supports each forecast. We finetune a supervisory language model that leverages this reasoning to jointly reconcile inconsistencies while improving the accuracy of related forecasts. On ForecastBench, our method improves the Brier score of a retrieval-augmented GPT-5.1 forecaster by 9.3% while reducing measures of structural inconsistency by 82-90%. We interrogate our method via a comprehensive series of baselines and ablations, and find that no alternative method matches our gains in consistency and accuracy simultaneously.
| Comments: | NeurIPS 2026 Workshop on Foundation Models for Temporal Systems (FMTS) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17735 [cs.LG] |
| (or arXiv:2611.17735v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17735 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kate Yuan [view email]
[v1]
Fri, 13 Nov 2026 04:29:15 UTC (42 KB)
来源:arXiv cs.AI · arxiv.org