HiLo：面向分布式在线后训练的高效 LoRA 同步框架
Train High, Ship Low: Efficient LoRA Synchronization for Distributed On-Policy Post-Training
AI 导读
arXiv 论文提出 HiLo，一个与压缩器无关的框架，用压缩梯度传播加本地 optimizer replay 取代 LoRA adapter 参数传输，保证 learner 与 rollout worker 策略状态一致。
正文 · 原文
Abstract:Transmitting LoRA adapters to rollout workers can bottleneck distributed on-policy post-training over bandwidth-constrained networks. Lowering the training rank may limit the capacity a workload needs, while adaptive optimizer updates are difficult to compress. Rollout workers can instead reconstruct updates from gradients by maintaining replicated optimizer states. We introduce HiLo, a compressor-agnostic framework that replaces parameter transmission with compressed gradient dissemination and local optimizer replay. By applying the same decoded gradients, the learner and workers maintain consistent policy states at the same version under reproducible execution. We further develop Spar, a budget-aware gradient compressor that combines spectral truncation, adaptive rank allocation and error feedback to decouple training rank from communication cost. Experiments across on-policy distillation and reward-guided post-training show that HiLo with Spar maintains learning quality close to full synchronization while transmitting as little as 1.6% of the adapter bytes, and accelerates end-to-end updates by up to $2.2\times$ under bandwidth constraints.
| Comments: | 19 pages, 7 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13306 [cs.LG] |
| (or arXiv:2611.13306v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13306 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shenghui Li [view email]
[v1]
Mon, 9 Nov 2026 22:18:34 UTC (1,654 KB)
来源：arXiv cs.AI · arxiv.org