跳到正文
arXiv cs.AI· Mridul Mahajan, Xuezhou Zhang, Aldo Pacchiano·· 2 小时前AI 评分38

分层策略如何在推理时分配采样预算:arXiv 新研究提升 pass@k

Learning Stratified Policies for Inference-Time Objectives

AI 导读

一项 arXiv 研究提出分层策略,用共享语言模型骨干上的多个 LoRA 适配器替代单一策略,把 k 个推理时采样预算分配给 k 个学习到的策略,并以 pass@k 为目标优化。理论分析表明,在单个 LoRA 均有正预测误差的线性设定下,LoRA 群体可实现零 best-of-k 近似误差。实验显示该方法在多个推理任务上的 pass@k 一致优于单策略基线。

正文 · 原文

View PDF HTML (experimental)

Abstract:Inference-time scaling improves the performance of large language models by sampling multiple responses and evaluating their collective utility. However, many existing approaches obtain these samples by repeatedly sampling from a single policy. In this work, we study if the sample budget at inference time can instead be allocated across a population of policies. We introduce stratified policies, where each of $k$ learned policies contributes one response to a $k$-sample inference-time objective. We realize this policy population using separate LoRA adapters on a shared language model backbone and optimize it for pass@$k$. We theoretically motivate the approach by analyzing the linear setting; a population of LoRAs can achieve zero best-of-$k$ approximation error in settings where every single LoRA has strictly positive prediction error. Empirically, we demonstrate that stratified policies consistently improve pass@$k$ over a single-policy baseline across multiple reasoning tasks. Overall, our results highlight how distributing samples across complementary policies can be a more effective use of sample budget than repeated i.i.d. sampling from a single policy at inference time.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23154 [cs.LG]
  (or arXiv:2611.23154v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.23154

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Mridul Mahajan [view email]
[v1] Tue, 17 Nov 2026 18:06:12 UTC (1,218 KB)

来源:arXiv cs.AI · arxiv.org