视觉基础模型空间状态表征探测研究
Probing Spatial State Representations in Visual Foundation Models
AI 导读
一项 arXiv 研究通过四级空间能力——空间结构、物体持续性、预测性空间更新和可行动性——在十个表征上对视觉基础模型进行 frozen-feature 探测，评估其能否维持持续空间状态。结果显示，可见场景读出强并不保证物体持续性或未来特征检索准确；目标依赖的候选选择也无法建立可靠的连续动作效用。物体持续性与可行动性建立在相机位姿读出之上，特征预测则建立在可见场景读出之上。
正文 · 原文
Abstract:An observer sees only part of the surrounding world, yet must keep track of what lies beyond the current view. Spatial perception therefore involves more than recognizing individual images: it calls for a persistent spatial state that is updated under self-motion and used to guide action. Do visual foundation models provide representations that support such a state? We study this question through four levels of spatial capability---spatial structure, object persistence, predictive spatial update, and actionability---using frozen-feature probes on ten representations under streaming egocentric observations of static indoor environments. The resulting capability profiles vary across tasks, network layers, and prefix lengths: strong readout of the visible scene does not ensure accurate object persistence or future-feature retrieval. Matched controls separate readout accuracy from reliance on visual history and motion. Native future prediction retains recoverable spatial structure, but a spatial reader trained on real features does not transfer to predicted features unchanged. Finally, goal-dependent candidate selection does not establish reliable continuous action utility: predicted actions respond to goal changes, yet supplying the correct goal does not reliably improve progress. Across the four levels, object persistence and actionability build on camera-pose readout, whereas feature prediction builds on visible-scene readout. Together, these findings motivate evaluating spatial representations not only by what can be decoded from them, but by whether that information is maintained beyond observation, updated predictively, and used for goal-directed action.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.40636 [cs.CV] |
| (or arXiv:2610.40636v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40636 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Baiqiao Yin [view email]
[v1]
Thu, 29 Oct 2026 17:58:16 UTC (2,373 KB)
来源：arXiv cs.AI · arxiv.org