arXiv cs.AI· Yucheng Guo, Yongjian Guo, Zhong Guan, Wen Huang, Haodong Yue, Shuai Di, Junwu Xiong, Yicheng Gong, Xi Xiao·· 4 小时前AI 评分26
D-VLA:面向视觉-语言-动作模型的高并发分布式异步强化学习框架
D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
AI 导读
研究者提出 D-VLA,一个面向大规模具身基础模型的高并发、低延迟分布式强化学习框架。该框架通过“平面解耦”隔离高频训练数据与低频权重控制,配合四线程异步“泳道”流水线、双池 VRAM 管理与拓扑感知复制,实现采样、推理、梯度计算与参数分发的全并行重叠。
正文 · 原文
Abstract:The rapid evolution of Embodied AI has enabled Vision-Language-Action (VLA) models to excel in multimodal perception and task execution. However, applying Reinforcement Learning (RL) to these massive models in large-scale distributed environments faces severe systemic bottlenecks, primarily due to the resource conflict between high-fidelity physical simulation and the intensive VRAM/bandwidth demands of deep learning. This conflict often leaves overall throughput constrained by execution-phase inefficiencies. To address these challenges, we propose D-VLA, a high-concurrency, low-latency distributed RL framework for large-scale embodied foundation models. D-VLA introduces "Plane Decoupling," physically isolating high-frequency training data from low-frequency weight control to eliminate interference between simulation and optimization. We further design a four-thread asynchronous "Swimlane" pipeline, enabling full parallel overlap of sampling, inference, gradient computation, and parameter distribution. Additionally, a dual-pool VRAM management model and topology-aware replication resolve memory fragmentation and optimize communication efficiency. Experiments on benchmarks like LIBERO show that D-VLA significantly outperforms mainstream RL frameworks in throughput and sampling efficiency for billion-parameter VLA models. In trillion-parameter scalability tests, our framework maintains exceptional stability and linear speedup, providing a robust system for high-performance general-purpose embodied agents.
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2605.13276 [cs.AI] |
| (or arXiv:2605.13276v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.13276 arXiv-issued DOI via DataCite |
Submission history
From: Yongjian Guo [view email]
[v1]
Wed, 13 May 2026 09:54:31 UTC (1,460 KB)
[v2]
Thu, 14 May 2026 06:59:41 UTC (1,460 KB)
[v3]
Fri, 13 Nov 2026 06:34:28 UTC (996 KB)
来源:arXiv cs.AI · arxiv.org