arXiv cs.AI· Ruiqi Shu, Xufeng Zhan, Yuanzhi Liang, Haibin Huang, Chi Zhang, Xuelong Li·· 4 小时前AI 评分43
TelePhysica:从单张照片进行物理交互场景的智能体重建
TelePhysica: Agentic Reconstruction of Physically Interactive Scenes
AI 导读
研究者提出 TelePhysica,一个闭环框架,从单张未标定照片重建可执行的、多模态的物理世界,用于有状态 3D 交互与基于轨迹的视频合成。视觉-语言编码智能体在两阶段验证循环中合成可分离网格与声明式物理模型,将单次合成 0% 的平衡验证场景恢复率提升至 81%。以模拟轨迹为条件驱动视频扩散模型,动态运动掩码重叠度提升 2×–5×,并在反事实力反转下保持运动方向。
正文 · 原文
Abstract:Direct image-to-video diffusion models entangle pixel synthesis with implicit latent motion, precluding persistent 3D mechanical equilibrium, mid-rollout intervention, and counterfactual force control. Conversely, single-pass programmatic 3D reconstruction from an uncalibrated photograph leaves monocular depth and support clearance underconstrained, causing reconstructed scenes to interpenetrate or collapse under gravity. We present TelePhysica, a closed-loop framework that reconstructs an executable, multi-regime physical world from a single photograph to drive stateful 3D interaction and trajectory-grounded video synthesis. A vision-language coding agent synthesizes separable visual meshes and declarative physical models within a two-stage verification loop that pairs pixel-aligned silhouette critique with a zero-force gravitational equilibrium preflight. The verified world compiles into a unified solver coupling rigid bodies, articulated chains, co-dimensional cloth shells, and 3D tetrahedral elastodynamics, whose simulated rollouts condition pretrained video diffusion models through pixel-aligned structural streams. On a 100-scene single-image benchmark, closed-loop execution raises equilibrium-verified world recovery from $0\%$ in single-pass synthesis to $81\%$. Controlled constitutive ablations show over an order-of-magnitude larger localized corner displacement and out-of-plane pleating in finite-element cloth than in matched rigid-plate proxies. Conditioning video diffusion on simulated trajectories increases dynamic motion-mask overlap by $2{\times}\text{--}5{\times}$ and preserves motion direction under counterfactual force reversals where direct image-to-video models fail.
| Comments: | Ongoing Work. Homepage: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.18078 [cs.CV] |
| (or arXiv:2611.18078v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.18078 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuanzhi Liang [view email]
[v1]
Fri, 13 Nov 2026 12:02:43 UTC (19,725 KB)
来源:arXiv cs.AI · arxiv.org