用空想性错视探究视觉模型的人脸选择性：Faces in Things 与 Faces in Nothing
Faces in Things, Faces in Nothing: Probing Face Selectivity in Vision Models with Pareidolia
AI 导读
一项研究用 FacesInThings 数据集测试六个视觉模型对人脸空想性错视（在插座等物体上看出脸）的反应。CLIP ViT-L/14 和两个生成式模型在类别被描述为人脸时约把一半这类区域标为 Human，而人类标注者读作动物或卡通脸；LLaVA-1.5 在移除脸状排列后减少 Human 判断，Qwen2-VL 则对平均色块也回答 Human 并描述出人。
正文 · 原文
Abstract:An electrical outlet can look like a face without depicting a person. Face pareidolia is thus a revealing probe of how vision models turn ambiguous visual evidence into category judgments. When a model calls such an object Human, does that answer depend on its face-like arrangement? Using the FacesInThings dataset, we test six vision models, from a vision-only ViT to CLIP and generative vision-language models, with face detectors as a reference. Each face-like region is paired with three controls from the same photograph that lack the arrangement: a background crop, the region with its tiles shuffled, and its average color. A Human answer driven by the arrangement should appear less often on these controls. We also vary how the categories are described. Three findings emerge. First, models respond to illusory faces, but not as people do: face detectors rarely return a face for these regions, yet CLIP ViT-L/14 and two generative models label roughly half of them Human when the categories are described as kinds of face. Annotators, by contrast, read these faces as non-human, such as animal or cartoon faces. For these models, a face is a human face by default. Second, similar Human rates hide opposite behavior. LLaVA-1.5 answers Human less often once the arrangement is removed; Qwen2-VL calls every average-color patch Human and describes a person in each; CLIP ViT-B/32 calls shuffled regions Human more often than real faces; SmolVLM rarely answers Human, but mostly for face-like regions. Third, rewording the categories changes whether Human answers favor the face at all, removing this preference in LLaVA-1.5 and creating it in CLIP. A Human answer is not evidence of a face: vision models can keep giving it after the face is gone.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.03989 [cs.CV] |
| (or arXiv:2603.03989v4 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2603.03989 arXiv-issued DOI via DataCite |
Submission history
From: Qianpu Chen [view email]
[v1]
Wed, 4 Mar 2026 12:33:36 UTC (3,687 KB)
[v2]
Tue, 21 Jul 2026 13:11:18 UTC (5,496 KB)
[v3]
Wed, 22 Jul 2026 15:07:20 UTC (2,412 KB)
[v4]
Wed, 4 Nov 2026 13:17:16 UTC (10,212 KB)
来源：arXiv cs.AI · arxiv.org