SearchHorizonBench：评测 AI 智能体搜索价值的基准
SearchHorizonBench: A Benchmark for Search Value in AI Agents
AI 导读
arXiv 论文提出 SearchHorizonBench，用于从执行经验中学习与评估 AI 智能体的搜索价值，共包含 3,168 个问题的 45,632 次执行。
正文 · 原文
Abstract:AI agents must decide when another search is worth its cost. Answer-success probabilities capture question difficulty, while this decision requires estimating the additional benefit of search. We introduce SearchHorizonBench, a benchmark for learning and evaluating search value from execution experience. Paired continuations and repeated executions separate expected search gain from generation noise and connect prediction quality to decision utility. The resource contains 45,632 executions across 3,168 questions. On 1,632 held-out questions, historical calibration reduces Brier error by 16.2%, with the improvement concentrated in success-level information. Search gains have a standard deviation of 21.4 percentage points, and tested historical estimators explain less than 2% of this variance in point estimates. Matched-configuration learning curves probe this distinction across training scales. At the reference token price, free selection observations yield a 2.49-point utility opportunity over the best constant action, driven primarily by gain information. Reusable histories, frozen splits, strong baselines and scoring tools provide a common foundation for developing and evaluating value-aware search agents.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.34714 [cs.AI] |
| (or arXiv:2610.34714v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34714 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xueqi Li [view email]
[v1]
Mon, 26 Oct 2026 02:04:38 UTC (275 KB)
来源：arXiv cs.AI · arxiv.org