arXiv cs.AI· Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin·· 3 小时前精选AI 评分61
Amazon 发布 Sandman 论文：共享 Kubernetes 上支撑智能体强化学习的沙箱服务
Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes
AI 导读
Amazon 团队在 arXiv 论文中介绍 Sandman，一个为智能体强化学习的工具执行与评估构建的沙箱服务，已在共享 Kubernetes 上支撑多个生产训练任务。
推荐理由
论文来自 Amazon Sandman 团队第一手，分享了共享 Kubernetes 上运行海量智能体 RL 沙箱的设计取舍与运维经验，还将开源 daemon、provider 和 pool。
正文 · 原文
Authors:Yiming Yan, Hongyang Wang, Wenbo Chen, Xuan Wang, Jingguo Liu, Xian Li, Chen Qi, Leo Fan, Siwei Li, Qianli Wu, Yichuan Li, Ruochen Wen, Qian Qu, Zheshen Wang, Benoit Dumoulin
Abstract:Agentic reinforcement learning (RL) often requires repeated tool calls in task-specific environments. A training job may create thousands of these environments in bursts while sharing a cluster with other jobs. We present Sandman, a sandbox service built and operated at Amazon for agent tool execution and evaluation. It serves multiple production agentic RL workloads on shared Kubernetes. A training framework supplies a task image and uses commands and file operations in the returned sandbox. Files and processes persist across calls until the framework releases the environment. Sandman manages creation, readiness and cleanup. Each sandbox is a Kubernetes pod with a small daemon mounted into the unmodified image. In a scale test, Sandman held 11,858 concurrent tiny sandboxes. Four production SWE-bench Verified training campaigns created 3.1 million sandboxes in October 2026. We report the design trade-offs and operational lessons from handling creation bursts, preparing task environments, reclaiming abandoned sandboxes, and running evaluation under controlled conditions. We will open-source Sandman's daemon, provider and pool.
| Comments: | 42 pages, 6 figures, 53 tables |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.39711 [cs.DC] |
| (or arXiv:2610.39711v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39711 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yiming Yan [view email]
[v1]
Thu, 29 Oct 2026 09:45:57 UTC (214 KB)
来源：arXiv cs.AI · arxiv.org