跳到正文
arXiv cs.AI· Jiachun Li, Zhuoran Jin, Kang Liu, Jun Zhao·· 2 小时前AI 评分36

MentalGrid 基准与 SpatialWalker:赋予多模态大模型心理空间模拟能力

From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs

AI 导读

研究者发布 MentalGrid 基准,评测多模态大模型在没有中间视觉反馈情况下模拟动态空间状态的能力,发现现有模型受限于感知不足和复杂场景的多步推理缺陷。团队提出三阶段课程学习框架,用 16k 空间推理样本训练出 SpatialWalker,较基座模型平均提升 33.8%,超越 Gemini-3-Flash,并在域外基准上保持 4.9% 平均增益。

正文 · 原文

View PDF HTML (experimental)

Abstract:Spatial reasoning is a fundamental component of world understanding for Multimodal Large Language Models (MLLMs). While existing research has advanced the perception of spatial relations in static scenes, the domain of dynamic spatial simulation remains relatively unexplored. To bridge this gap, we introduce MentalGrid, a benchmark focused on spatial reasoning tasks that require models to simulate evolving spatial states without intermediate visual feedback. Extensive evaluation on our benchmark reveals a significant deficiency in the spatial simulation capabilities of existing models. Specifically, we find that performance is constrained by perceptual limitations in simple simulation scenarios and hindered by inadequate multi-step reasoning as complexity increases. To address these challenges, we propose a three-stage curriculum learning framework utilizing 16k specialized spatial reasoning samples. This approach yields SpatialWalker, which achieves a 33.8% average improvement over its base model, surpassing proprietary models like Gemini-3-Flash in spatial simulation. Furthermore, SpatialWalker demonstrates robust generalization, maintaining a 4.9% average gain on out-of-domain benchmarks, demonstrating the pivotal role of spatial simulation as a fundamental building block for broader spatial intelligence.
Comments: Findings of EMNLP 2026
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2611.22541 [cs.CV]
  (or arXiv:2611.22541v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.22541

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jiachun Li [view email]
[v1] Tue, 17 Nov 2026 10:31:56 UTC (1,236 KB)

来源:arXiv cs.AI · arxiv.org