用视觉-语言模型嵌入做图像聚类：Qwen3-VL 与 BLIP-2 的 252 组基线对比研究
Image Clustering with Vision-Language Model Embeddings
AI 导读
一项研究在七个图像基准上比较九种表征流水线、四种聚类配置，共 252 组基线组合，评估视觉-语言模型嵌入用于图像聚类的效果。固定 Ward 层次聚类下，Qwen3-VL 在五个数据集上取得最高调整兰德指数（ARI），MNIST 上达 0.925；BLIP-2 在 CIFAR-10 和 CIFAR-100 上分别取得 0.925 和 0.591。
正文 · 原文
Abstract:Pretrained vision-language models provide image representations whose suitability for clustering may vary with feature extraction, image domain, and partitioning algorithm. We compare nine representation pipelines across seven image benchmarks and four clustering configurations, yielding 252 baseline combinations. Eight pipelines extract visual features, including two vision-only references; the ninth generates descriptions and embeds the resulting text. With agglomerative clustering (Ward linkage) fixed, Qwen3-VL achieves the highest adjusted Rand index (ARI) on five datasets, including 0.925 on MNIST, while BLIP-2 leads on CIFAR-10 and CIFAR-100 with ARIs of 0.925 and 0.591, respectively. A separate sensitivity analysis compares raw and L2-normalized embeddings at their original dimensionality across all 63 dataset-representation pairs. Normalization preserves the highest-ARI representation in every dataset, increases ARI in 35 pairs, decreases it in 26, and leaves two numerically unchanged. On the SVHN dataset, the highest ARI remains low, rising from 0.097 to 0.098 after normalization. The evaluated caption-mediated pipeline obtains relatively high ARI on isolated digits but does not consistently exceed direct visual features. These findings support selecting and evaluating complete representation-clustering pipelines for the intended domain, with explicit attention to extraction and normalization choices.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33265 [cs.CV] |
| (or arXiv:2610.33265v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33265 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: João Pedro Matos-Carvalho [view email]
[v1]
Fri, 23 Oct 2026 20:21:15 UTC (3,354 KB)
来源：arXiv cs.AI · arxiv.org