BunshinRL：面向智能体强化学习 Rollout 的全工作区分叉系统
BunshinRL: Whole-Workspace Forking for Agentic Reinforcement Learning Rollouts
AI 导读
BunshinRL 是一套智能体强化学习 rollout 系统，通过跨环境内存共享、工具前缀缓存和投机式工具执行三项技术降低并发环境开销，其基础是容器级实况全工作区分叉机制 BunshinFork。系统通过 Linux 内核模块和 CRIU 修改实现，可将峰值内存最多降低 43%、总内存消耗最多降低 51%，端到端 rollout 延迟降低 23%。
正文 · 原文
Abstract:Agentic reinforcement learning (RL) rollouts interleave model reasoning with tool calls in persistent workspaces. Scaling these rollouts incurs substantial memory costs for concurrent environments, while slow tool calls delay entire rollout batches. We present BunshinRL, an agentic RL rollout system that exploits shared environment state and recurring tool calls through three techniques: memory sharing across environments, tool-prefix caching, and speculative tool execution. These techniques build on BunshinFork, a novel live whole-workspace forking mechanism for containers. BunshinFork reconstructs process trees, shares memory copy-on-write, and preserves file, communication, and GUI state, enabling cached-state reuse and rollback after incorrect speculation. We implement BunshinRL and BunshinFork through Linux kernel module and CRIU modifications. BunshinRL reduces peak memory by up to 43%, total memory consumption by up to 51%, and end-to-end rollout latency by 23%.
| Subjects: | Artificial Intelligence (cs.AI); Operating Systems (cs.OS) |
| Cite as: | arXiv:2611.10453 [cs.AI] |
| (or arXiv:2611.10453v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.10453 arXiv-issued DOI via DataCite |
Submission history
From: Yiying Zhang [view email]
[v1]
Sat, 7 Nov 2026 04:05:17 UTC (498 KB)
[v2]
Wed, 11 Nov 2026 06:00:19 UTC (498 KB)
来源：arXiv cs.AI · arxiv.org