arXiv cs.AI· Tianjun Gu, Jingyu Gong, Zhizhong Zhang, Yuan Xie, Lizhuang Ma, Xin Tan, Athanasios V. Vasilakos·· 4 小时前AI 评分47
视觉语言模型在物理动力学与意图推理上落后于人类表现
Vision-language models lag human performance on physical dynamics and intent reasoning
AI 导读
一项研究提出 Teleo-Spatial Intelligence(TSI)概念,并构建基于 11,328 段真实视频的 EscherVerse 资源,包含 8,000 例基准和 35,963 例指令微调集。
正文 · 原文
Abstract:Spatial intelligence is central to embodied cognition, yet contemporary AI systems still struggle to reason about physical interactions in open-world human environments. Despite strong performance on controlled benchmarks, vision-language models often fail to jointly model physical dynamics, reference frames, and the latent human intentions that drive spatial change. We introduce Teleo-Spatial Intelligence (TSI), which captures an important aspect of spatial reasoning: integrating physical dynamics and action goals to reason about actions and their spatial consequences. To evaluate TSI, we present EscherVerse, a large-scale open-world resource built from 11,328 real-world videos, including an 8,000-example benchmark and a 35,963-example instruction-tuning set. Across 27 state-of-the-art vision-language models and an independent analysis of first-pass human responses from 11 annotators, we identify a persistent teleo-spatial reasoning gap: the strongest proprietary model achieves 57.26% overall accuracy, far below first-pass human performance, which ranges from 84.81% to 95.14% with a mean of 90.62%. Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it. EscherVerse provides a diagnostic testbed for purpose-aware spatial reasoning and highlights a critical gap between pattern recognition and human-level understanding in embodied AI.
| Comments: | 29 pages, 7 figures. Supplementary Information and Source Data are included as ancillary files |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.01547 [cs.CV] |
| (or arXiv:2601.01547v3 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2601.01547 arXiv-issued DOI via DataCite |
Submission history
From: Tianjun Gu [view email]
[v1]
Sun, 4 Jan 2026 14:42:39 UTC (10,950 KB)
[v2]
Mon, 23 Mar 2026 04:28:17 UTC (21,008 KB)
[v3]
Fri, 13 Nov 2026 09:10:02 UTC (13,195 KB)
来源:arXiv cs.AI · arxiv.org