MAC-v2：从像素实现长时程基于模型的价值扩展
MAC-v2: Long-horizon model-based value expansion from pixels
AI 导读
研究团队提出 MAC-v2，在基于像素的离线强化学习中实现长时程基于模型的价值扩展。该方法训练无重构的潜变量世界模型并结合动作分块，从预测与控制目标学习表示，减少递归预测误差累积。在 visual OGBench、MMBench2、Robomimic 和 LIBERO-10 等视觉基准上，MAC-v2 性能优于 model-free 和 model-based 离线 RL 基线。
正文 · 原文
Abstract:We study whether \emph{long-horizon model-based value expansion} can serve as an effective framework in pixel-based offline reinforcement learning (RL). Long-horizon model-based value expansion from pixels is substantially harder than state-based environments, as the agent must learn both the representation and its dynamics, and errors from either can accumulate recursively over imagined trajectories. We introduce MAC-v2, which enables long-horizon model-based expansion from pixels by training a reconstruction-free latent world model with action chunking. By learning representations from prediction and control objectives, MAC-v2 does not need to encode control-irrelevant visual details that can hinder the dynamics prediction. Action chunking further mitigates the prediction error by reducing the number of recursive model predictions required for value expansion. Across diverse visual benchmarks including visual OGBench, MMBench2, Robomimic, and LIBERO-10, MAC-v2 achieves strong performance against model-free and model-based offline RL baselines. Our analyses show that long-horizon model-based value expansion is critical for the performance of MAC-v2, and that both action chunking and latent learning objectives are essential for reliable long-horizon latent imagination.
| Comments: | 9 pages (12 pages of appendix & references), submitted to ICLR 2027 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07245 [cs.LG] |
| (or arXiv:2611.07245v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07245 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kwanyoung Park [view email]
[v1]
Wed, 4 Nov 2026 23:51:02 UTC (221 KB)
来源：arXiv cs.AI · arxiv.org