研究测量七个视觉语言模型读取大图小细节的能力并给出何时放大的五条规则
Know When to Zoom: How Small Is Too Small for Vision-Language Models?
AI 导读
arXiv 论文 arXiv:2611.22709 测量四个厂商七个在线视觉语言模型对大图小细节的读取能力,发现 8000×6000 无人机照片中 20 像素宽的物体到 Claude 处只剩约 6 像素,OpenAI 默认设置会直接拒收。
正文 · 原文
Abstract:Online vision-language models often miss small details in large images, because the service shrinks the image, or rejects it, before the model ever sees it. A car 20 pixels wide in an 8000 by 6000 drone photo reaches Claude about 6 pixels wide, and OpenAI's default rejects the photo outright. We measure how much detail survives for seven models from four vendors, using an eye exam of printed codes and 601 questions about small objects in real photos. The smallest text a model reads half the time grows from 6 pixels or less on a small image to a predicted 10 to 13 pixels on a 7680 by 4320 photo at default settings. Zooming wins much of the detail back. Splitting each photo into up to nine tiles raises accuracy by 11.1 to 14.5 percentage points for Claude and Gemini, and by avoiding rejections it lifts GPT-6.1 Sol from 50.7% to 83.7%. A two-step zoom, in which the model first names the region it needs, matches or beats tiles on Gemini and comes within 2.5 points of them on Claude, with about 40% of their image tokens. When a detail is lost, Claude and Gemini both say they cannot tell about half the time, but they fail differently in the rest: Gemini gives a confident wrong answer in 43% to 48% of these cases, Claude in only 5% to 9%. A check based on detail size, run before the call, misses most of the gain from tiles on Claude and Gemini, but Claude's own reply is a reliable trigger: tiling only after an unsure reply comes within 1.4 points of always tiling with 62% to 70% of the image tokens. We turn these findings into five rules for when an agent should zoom, and release the eye exam, every model answer and code at this https URL.
| Comments: | Accepted to the SENSE workshop at WACV 2027. 10 pages, 5 figures, 3 tables. Code, data and a plain-language walkthrough: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22709 [cs.CV] |
| (or arXiv:2611.22709v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22709 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Siddharth Vohra [view email]
[v1]
Tue, 17 Nov 2026 12:50:14 UTC (179 KB)
来源:arXiv cs.AI · arxiv.org