arXiv cs.AI· Samuele Punzo, Niccol\`o Caselli, Ippokratis Pantelidis, Francesco Massafra, Salvatore Lo Sardo, Mohammadreza Salehi·· 5 小时前AI 评分43
视频基础模型如何编码直觉物理？跨预训练范式的探测研究
How Do Video Foundation Models Encode Intuitive Physics? Probing Across Pretraining Paradigms
AI 导读
一项研究用冻结特征探测方法，检验视频基础模型是否在表征中编码直觉物理信息，比较了联合嵌入模型 V-JEPA、掩码重建模型 VideoMAE 和扩散视频生成模型 LTX-Video。
正文 · 原文
Abstract:Do video foundation models encode a usable sense of how the physical world works, and does intuitive physics emerge naturally from large-scale video pretraining? In this work, we study whether pretrained video foundation models encode intuitive-physics information in their frozen representations, and how its accessibility varies across pretraining paradigms, layers, backbone variants, and probe types. Using frozen-feature probing on IntPhys2 and Minimal Video Pairs (MVP), we compare predictive joint-embedding models (V-JEPA), masked reconstruction models (VideoMAE), and diffusion-based video generators (LTX-Video). We find that across linear, MLP, and temporal attentive probes, physics-relevant information is generally weakest in early layers and most accessible at intermediate to late depth. On MVP, temporal attentive probes substantially outperform linear readouts, with V-JEPA achieving 95.0% pair consistency. On IntPhys2, linear probes recover more signal, and VideoMAE attains the strongest temporal attentive result at 73.9% violation-of-expectation accuracy. On both benchmarks, attentive probe performance on LTX consistently underperforms compared to the other two pretraining paradigms. Together, these results suggest that intuitive-physics is broadly decodable from video foundation models, but it is not represented as a single, uniform capability. Its accessibility, and especially its temporal grounding, depends much more on the benchmark and readout than on a simple hierarchy of pretraining objectives or model scale.
| Comments: | Accepted at the NeurIPS 2026 Workshop on World Models in Physical AI |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.09646 [cs.CV] |
| (or arXiv:2606.09646v3 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.09646 arXiv-issued DOI via DataCite |
Submission history
From: Niccolò Caselli [view email]
[v1]
Mon, 8 Jun 2026 15:40:32 UTC (1,755 KB)
[v2]
Sun, 13 Sep 2026 15:50:40 UTC (2,038 KB)
[v3]
Fri, 30 Oct 2026 08:34:47 UTC (2,022 KB)
来源：arXiv cs.AI · arxiv.org