arXiv cs.AI· Nikos Giakoumoglou, Paschalis Giakoumoglou, Andreas Floros, Kleanthis Marios Papadopoulos, Tania Stathaki·· 6 小时前AI 评分37
SNAP：合成难负样本为何在视觉-语言预训练中有效
What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?
AI 导读
arXiv 论文分析六种表示空间合成策略在视觉-语言预训练中的失效模式：跨模态构造产生过易负样本或被拉向查询，模态内构造会混入匹配正样本，并提出 SNAP 方法避免这两种失效。SNAP 不依赖外部生成模型，与模型无关，训练时间开销低于 10%。在 CLIP 和 FLIP 上评估，SNAP 在零样本检索、零样本分类和 linear probe 上取得一致提升。
正文 · 原文
Abstract:Synthetic hard negatives generated in the representation space have proven effective for unimodal self-supervised learning, but transferring this idea to vision-language pretraining is not straightforward. We analyze six representation-space synthesis strategies and identify two failure modes in their transfer to vision-language pretraining: cross-modal constructions that produce overly easy negatives or pull them toward the query, and intra-modal constructions that incorporate the matched positive. We also observe logit-scale saturation when training with synthetic hard negatives and a learnable temperature, and find that fixing the temperature improves downstream performance. Using this geometric analysis we propose SNAP, which generates intra-modal hard negatives that never involve the positive from either modality, avoiding both failure modes entirely. SNAP is model-agnostic, requires no external generative models, and adds less than 10% training time overhead. Evaluated on top of CLIP and FLIP across multiple architectures and datasets, SNAP delivers consistent improvements on zero-shot retrieval, zero-shot classification, and linear probe evaluation.
| Comments: | ACCV 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.09700 [cs.CV] |
| (or arXiv:2610.09700v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.09700 arXiv-issued DOI via DataCite |
Submission history
From: Nikos Giakoumoglou [view email]
[v1]
Wed, 7 Oct 2026 08:58:57 UTC (1,498 KB)
[v2]
Mon, 26 Oct 2026 11:44:50 UTC (1,720 KB)
来源：arXiv cs.AI · arxiv.org