CueClash 基准测试:前沿视觉语言模型判断贴近画面平面的三维关系
CueClash: Frontier Vision-Language Models Judge 3-D Relations Close to the Picture Plane
AI 导读
arXiv 论文提出 CueClash 渲染场景基准,将画面二维线索与地面布局分离,测试 Claude Opus 5.5、GPT-5.5 和 Gemini 3.6 Flash 判断三维关系的能力。
正文 · 原文
Abstract:A vision-language model (VLM) can judge 3-D relations from the picture or from the ground layout. In photographs the two usually agree. We separate them with CueClash, rendered scenes in which the 2-D cue for each of five relations is right on exactly half the statements. The shortcut gap is accuracy when the cue is right minus accuracy when it is wrong. For Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash at low reasoning effort it is +18 to +37 percentage points. It is +7 to +29 without two relations whose cue-wrong items are confounded with borderline cases or occlusion. Given ground coordinates and each relation's definition, they answer 99 to 100% correctly. Pixel measurements with camera depths remove most of the gap for objects fitting through openings. The failure lies in the step from the picture to the layout. The maps that the models draw of the scene from above mostly keep the depth order but, like the picture, shorten depth. GPT-5.5's maps stay close to a copy of the picture. Opus 5.5's and Gemini's maps undo about 0.6 of this shortening. Gemini's then shrink all depths by a further uniform factor. Opus 5.5 follows the picture mainly on fits, and its maps correct depth about as much as human observers of real scenes do. The three models also lose accuracy on CV-Bench photographs where a reliable 2-D cue misleads. In two open models the layout information reaches the language model, but their maps lose much of it.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2611.22180 [cs.CV] |
| (or arXiv:2611.22180v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22180 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Soham Dan [view email]
[v1]
Tue, 17 Nov 2026 04:17:35 UTC (1,693 KB)
来源:arXiv cs.AI · arxiv.org