跳到正文
arXiv cs.AI· Wenbo Zhu, Paul Yi-Chia Chen, Yuanhan Li, Zishen Wan, Christopher Torng·· 2 小时前AI 评分39

制造差分稀疏性:面向高效世界模型规划的规划器-硬件协同设计

Manufacturing Differential Sparsity: Planner-Hardware Co-Design for Efficient World Model Planning

AI 导读

arXiv 论文提出一种规划器与硬件协同设计方法,针对 JEPA 风格潜在世界模型的 MPC 规划过慢问题,利用预测器的潜在局部性制造差分位级稀疏性,通过参考森林采样将同组动作序列耦合到共享根节点。在三个 JEPA 世界模型上,该设计比稠密加速器快 1.28-1.80 倍、节能 38-52%,比先前位稀疏设计快 1.22-1.58 倍、节能 16-36%,面积相同。

正文 · 原文

View PDF HTML (experimental)

Abstract:JEPA-style latent world models let embodied agents plan toward new goals with model predictive control, but they are still too slow for real-life embodied control: sampling-based MPC repeats predictor inference tens of thousands of times per plan, and even on high-end GPUs one planning step can take tens of seconds. We observe substantial latent locality in this repeated-inference workload: the predictor maps close actions to close latents, and inside a quantized predictor this closeness takes a form that hardware can exploit, differential bit-level sparsity. Unlike an input stream, whose temporal similarity is given, a planner itself decides which candidates it evaluates together, so the planner can choose its own references and, with reference forest sampling, manufacture more sparsity: it couples the sampled action sequences of each group to a shared root, with competitive planning quality. We build the accelerator on the insights of the rich line of bit-level sparsity accelerators and design it and the planner tightly coupled as a single co-design: the planner increases sparsity and exposes structure for hardware scheduling, and the bit-exact hardware effectively uses the sparsity that the planner creates. On three JEPA world models, our co-design is 1.28-1.80x faster than a dense accelerator and uses 38-52% less energy, and it is 1.22-1.58x faster than prior bit-sparse design and uses 16-36% less energy, with the same area. To our knowledge, our planner is the first planner that manufactures differential bit-level sparsity across its candidates and exposes its structure for hardware scheduling, and our co-design is the first algorithm-hardware co-design for JEPA-style world-model planning.
Comments: 14 pages, 11 figures
Subjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO); Systems and Control (eess.SY)
Cite as: arXiv:2611.22483 [cs.AR]
  (or arXiv:2611.22483v1 [cs.AR] for this version)
  https://doi.org/10.48550/arXiv.2611.22483

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Wenbo Zhu [view email]
[v1] Tue, 17 Nov 2026 09:40:21 UTC (1,161 KB)

来源:arXiv cs.AI · arxiv.org