arXiv cs.AI· Rahim Hossain, Dipanwita Saha, Ashirbad Mishra, Xinxin Shu, Svetlana Strunjas, Binbin Li·· 56 分钟前AI 评分34
LoCaR:基于冻结视觉语言模型的局部画布检索框架
LoCaR: Localized Canvas Retrieval With Frozen Vision-Language Model
AI 导读
LoCaR 是一个免训练的局部文本到图像检索框架,利用冻结视觉语言模型,通过 SAM2 生成的独立候选区域对每个区域单独编码,获得更局部化的表示;还提供成对样本适配作为可选残差,不更新底层编码器。论文同时推出基于 Visual Genome 的 LOCFG 基准,用于评测更小、更具挑战性的指称对象的局部检索,结果显示区域表示本身的改进是主要收益来源。
正文 · 原文
Abstract:Whole image embeddings can under represent small objects and local attributes that distinguish relevant images for localized queries. We present LoCaR, a training free framework for localized text to image retrieval with frozen vision language models. LoCaR constructs and utilizes independent proposals (SAM2) focusing on the object so as to get a more localized representation that independently encodes each region using the frozen visual encoder. We further study paired-support adaptation as an optional residual that uses matched text region exemplars without updating the underlying encoders. We also introduce LOCFG, a Visual Genome derived benchmark designed to evaluate localized retrieval for smaller and more challenging referents. Across several vision language backbones, our results show that improving the region representation itself is the main source of gain.
| Comments: | Zero-shot retrieval , Local Retrieval |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.22131 [cs.AI] |
| (or arXiv:2611.22131v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22131 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rahim Hossain [view email]
[v1]
Tue, 17 Nov 2026 03:15:02 UTC (10,231 KB)
来源:arXiv cs.AI · arxiv.org