arXiv cs.AI· Jiaqi Wu, Yongqi Dong, Yijie Zhong, Maple Shao, Yide Tao, Linsen Cai, Weize Cai, Hai L. Vu, Alvaro Garcia Hernandez·· 2 小时前AI 评分30
残差引导轻量级强化学习方法实现风险感知轨迹选择,采用冻结世界模型
Residual-Guided Lightweight Reinforcement Learning for Risk-Aware Trajectory Selection with a Frozen World Model
AI 导读
一项自动驾驶研究提出残差引导的强化学习轨迹选择框架,用 PPO 微调候选轨迹选择策略,配合冻结世界模型和固定候选库。该方法仅微调 16,106 个参数,部署时仅保留 11,725 参数的排序网络;在 nuScenes 验证集开环评估中,相比无强化学习的 RiskWorld 基线将平均 L2 轨迹误差降低 3.99%,残差探索在同等预算下进一步降低 0.83%。
正文 · 原文
Abstract:In automated driving, world models provide future scene predictions for evaluating candidate trajectories, but fixed selection rules may not capture scene-dependent trade-offs. We propose a residual-guided reinforcement learning (RL) framework for lightweight trajectory selection with a frozen world model and a fixed candidate bank. The framework uses proximal policy optimization (PPO) to fine-tune a joint policy over candidate selection and bounded trajectory residuals. A context-conditioned ranking network learns candidate preferences from feedback obtained by evaluating local perturbations of the selected trajectories. At inference, the selector combines learned preference and predicted risk ranks using fixed weights and returns the selected candidate trajectory without modification. Fine-tuning updates 16,106 parameters across the actor and critic, while the deployed selector retains only the 11,725-parameter ranking network, with the residual branch and critic used exclusively during training. In open-loop evaluation on the nuScenes validation set, the method reduces average Euclidean trajectory error (L2) by 3.99% relative to the RiskWorld model baseline without RL-based selection. Under matched budgets for feedback accesses and actor updates, residual exploration reduces L2 by 0.83% relative to the categorical PPO variant trained without residuals. The method also maintains lower L2 than RiskWorld when risk features are removed or shuffled.
| Comments: | 6 pages, 2 figures |
| Subjects: | Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET); Machine Learning (cs.LG); Robotics (cs.RO); Systems and Control (eess.SY) |
| Cite as: | arXiv:2611.22998 [cs.AI] |
| (or arXiv:2611.22998v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22998 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yongqi Dong [view email]
[v1]
Tue, 17 Nov 2026 16:12:55 UTC (17,982 KB)
来源:arXiv cs.AI · arxiv.org