AutoPipe：LLM 后训练流水线的自动配置框架
Automatic Configuration of LLM Post-Training Pipelines
AI 导读
研究者提出 AutoPipe，一个面向 LLM 后训练（SFT 与强化学习）流水线的预算感知两阶段自动配置框架。离线阶段从历史运行学习数据集条件的学习排序代理模型，在线阶段引导贝叶斯优化并用高斯过程残差代理建模数据集差异，同时通过早停和预测器降低评估成本。在生物医学推理任务上，AutoPipe 以远低于全保真 HPO 基线的在线成本取得相当性能，并优于纯离线方法和早停 HPO 变体。
正文 · 原文
Abstract:LLM post-training pipelines that combine supervised fine-tuning and reinforcement learning are difficult to configure under realistic compute budgets: the configuration space is high-dimensional and heterogeneous, stages are strongly coupled, and each end-to-end evaluation is expensive. We propose AutoPipe, a budget-aware two-phase framework for configuration selection in LLM post-training. Offline, AutoPipe learns a dataset-conditioned learning-to-rank surrogate from historical runs, capturing within-dataset preferences and providing transferable guidance toward promising regions of the configuration space. Online, for a new dataset, AutoPipe uses the offline guidance to steer Bayesian optimization and models dataset-specific deviations with a Gaussian-process residual surrogate. To reduce evaluation cost, each trial is early-stopped and scored by a learned predictor that maps early training signals to a low-cost proxy for final post-training performance. Experiments on biomedical reasoning tasks show that AutoPipe achieves performance comparable to full-fidelity HPO baselines at a fraction of their online cost, while outperforming offline-only approaches and early-stopped HPO variants.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.18773 [cs.LG] |
| (or arXiv:2603.18773v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.18773 arXiv-issued DOI via DataCite |
Submission history
From: Channe Chwa [view email]
[v1]
Thu, 19 Mar 2026 11:26:56 UTC (107 KB)
[v2]
Sun, 25 Oct 2026 06:30:57 UTC (130 KB)
来源：arXiv cs.AI · arxiv.org