跳到正文
arXiv cs.AI· Jia Hu, Yang Chang, Panqu Wang, Zhijun Fu, Chang Liu, Changjian Yu, Jintao Lai·· 2 小时前AI 评分38

面向自动驾驶的闭环交互式世界模型让环境真实反应

Making the World React Realistically: A Closed-Loop Interactive World Model for Autonomous Driving

AI 导读

一篇 arXiv 论文提出用于自动驾驶的闭环交互式世界模型,将行为建模与视觉渲染耦合。方法通过多智能体强化学习微调模仿学习的背景车辆策略,并对每个 3D Gaussian 车辆资产做逐 Gaussian 诊断以修正未观测视角,同时根据车辆驾驶意图渲染信号灯。

正文 · 原文

View PDF HTML (experimental)

Abstract:World models (WMs) generate driving scenarios beyond recorded logs, enabling the training and evaluation of autonomous driving systems in unseen interactions. However, existing WMs struggle to ensure both behavioral realism and visual fidelity, since they overlook the relationship between vehicle interactions and the ego visual observations. We propose a closed-loop interactive WM that couples behavior modeling and visual rendering. First, imitation-learned background-vehicle (BV) policies are fine-tuned with multi-agent reinforcement learning in the closed loop against an end-to-end ego planner that reacts to the rendered images, so that their interaction behaviors match those in real logs. Second, the viewpoints exposed by these interactions guide a per-Gaussian diagnosis of each 3D Gaussian vehicle asset, which attenuates view-dependent colors on under-observed Gaussians and fills never-observed surfaces with generated Gaussians, while preserving the observed appearance. Third, signal lights are rendered according to the driving intentions of the vehicles, by an editor that learns the light states unobserved for a vehicle through counterfactual fine-tuning with a light-state classifier. On long-tail closed-loop scenarios, our method reduces the divergence from real-world interaction behaviors by 85% and improves novel-view vehicle PSNR by 2.0dB over the state-of-the-art WM. The significance of BV light signaling in WMs is shown, as the rendered signal lights conveying BV driving intentions indirectly impact ego driving behaviors in a quarter of the scenarios.
Comments: 11 pages, 5 figures
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22691 [cs.RO]
  (or arXiv:2611.22691v1 [cs.RO] for this version)
  https://doi.org/10.48550/arXiv.2611.22691

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yang Chang [view email]
[v1] Tue, 17 Nov 2026 12:39:42 UTC (7,716 KB)

来源:arXiv cs.AI · arxiv.org