arXiv cs.AI· Jialiang Wang, Qing Ma, Xiong Zhou, Gangfeng Hu, Haoliang Li, Xianming Liu·· 3 小时前AI 评分32
对比学习的内部差异：泛化界与下游行为研究
Internal Discrepancy for Contrastive Learning: Generalization Bounds and Downstream Behaviors
AI 导读
一篇 arXiv 论文提出“内部差异”（internal discrepancy）度量，量化正样本平均表示与潜在聚类中心的差异，并证明感知该度量的下游监督任务泛化界。该泛化界统一了已有多种理论假设，并据此引入松弛 k-means 损失作为近似以降低内部差异。论文还从对比学习诱导的监督损失视角分析下游行为，解释类别不平衡场景下对比学习促进不平衡学习的原因。
正文 · 原文
Abstract:Contrastive learning has emerged as a popular paradigm for training representation models. While prior studies pursue more practical and less restrictive assumptions to enhance theoretical understanding of contrastive learning, they essentially describe the interconnections among representations within the data or induced by the function class. To encapsulate these assumptions, we propose the measure of internal discrepancy that quantifies the difference between the mean representation of positive samples and latent cluster centers. We then prove an internal discrepancy-aware generalization bound for downstream supervised tasks following contrastive learning, without requiring additional assumptions. This generalization bound not only serves as a unifying framework connecting various assumptions but also motivates us to further reduce the internal discrepancy, thereby introducing the relaxed k-means loss as an approximation. Moreover, we investigate the downstream behaviors of contrastive learning from the perspective of contrastive learning-incurred supervised losses. In the presence of class imbalance, we shed light on why contrastive learning can facilitate imbalanced learning. Finally, we investigate the optimality of these incurred supervised losses, proving several intriguing properties.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32729 [cs.LG] |
| (or arXiv:2610.32729v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32729 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jialiang Wang [view email]
[v1]
Fri, 23 Oct 2026 14:48:35 UTC (320 KB)
来源：arXiv cs.AI · arxiv.org