arXiv cs.AI· Jalaj Bhandari, Dhawal Gupta, Hongli Zhan, Zhenting Wang, Varad Pimpalkhute, Zhengzhong Liu, Eric Xing, Rupesh Srivastava·· 3 小时前AI 评分40
论 turn-level Best-of-N 采样如何为 LLM 智能体扩展推理
On Turn-Level Best-of-N Sampling for Scaling LLM Agents
AI 导读
一项研究评估了面向 LLM 智能体的 turn-level best-of-N 推理时扩展方法：在每一轮中由 generator 模型生成 N 个候选动作，selector 模型从中选一个执行。在长程终端任务上的实验显示，该方法可将准确率提升 5–10%，并提高重复尝试间的一致性。轨迹分析显示智能体行为向更优模式转变，如减少解题尝试次数、更好地遵循任务指令以及调试和测试。
正文 · 原文
Abstract:In this work, we evaluate a simple inference-time scaling approach based on \textit{turn-level best-of-N sampling} (\method) for large language model (LLM) agents on complex, long-horizon tasks. We instantiate this using a \textit{generator-selector} framework with two LLMs; at each turn, a \emph{generator} model generates $N$ candidate actions and a \emph{selector} model selects one of them, which is then executed in the environment. This paper is closely related to the work of \cite{zainullina2025guided} who used critic models for guided search at inference time. Different from their work, we evaluate if off-the-shelf, open-weight LLMs can be effective at comparing different candidate actions directly. Experiments on long-horizon terminal tasks show that \method can lead to significant performance improvements, both in terms of accuracy (5--10\%) and consistency across repeated attempts. Analysis of agent trajectories reveal a shift in agent behaviour towards desirable patterns such as reduction in number of attempts to solve the task, improved task instruction following, debugging and testing.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.06868 [cs.AI] |
| (or arXiv:2611.06868v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06868 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jalaj Bhandari [view email]
[v1]
Wed, 4 Nov 2026 18:23:14 UTC (1,179 KB)
来源：arXiv cs.AI · arxiv.org