SceneAgent：从 3D 与 4D 采集到仿真环境的智能体流水线
SceneAgent: An Agentic Pipeline from 3D and 4D Captures to Simulation Environments
AI 导读
SceneAgent 是一条智能体流水线，通过基于仿真器的视觉检查与修复，把 3DGS、摄影测量、LiDAR、AI 生成 3DGS 场景和 4D 动态视频转化为可交互仿真环境。
正文 · 原文
Abstract:Visual captures provide detailed scene appearance, but converting them into interactive simulation environments requires object reconstruction, physical modeling, and correction of intermediate errors. We present SceneAgent, an agentic pipeline that coordinates these steps through simulator-based visual inspection and repair. The pipeline processes 3D Gaussian Splatting (3DGS), photogrammetry, LiDAR captures, AI-generated 3DGS scenes, and 4D videos of dynamic scenes, and combines object segmentation, physical parameter estimation, part decomposition, articulation, and deformable simulation. To support policy learning, SceneAgent generates diverse object variants ("digital sisters") and scripted demonstrations. On a cluttered bin scene from GraspClutter6D, SceneAgent raises F1 at 1 cm for every coding agent with scorable outputs both with and without its skills, and its reconstructions are better than SimFoundry: all agent configurations exceed SimFoundry's F1 score, with a best of 60.75% versus 6.9%. On three dynamic scenes from 4DCodeBench, SceneAgent raises the mean dynamic IoU of Gemini 4 Argon from 0.86% to 99.09%. We further demonstrate diverse simulated interactions and preliminary policy transfer to a real robot.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2610.34745 [cs.RO] |
| (or arXiv:2610.34745v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34745 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Luke Hollis [view email]
[v1]
Mon, 26 Oct 2026 02:45:35 UTC (29,450 KB)
来源：arXiv cs.AI · arxiv.org