长尾数据下监督对比学习表征几何的重新思考
Rethinking Representation Geometry in Supervised Contrastive Learning under Long-Tailed Data
AI 导读
一项研究提出理论框架，分析监督对比学习在长尾分布下最优表征是否形成正则单纯形几何，发现现有目标函数因依赖每类样本数而无法在最优处形成该几何。研究者据此提出结构性平衡方法与完全平衡的对比目标，可证明在最优处形成正则单纯形。理论与实验结果显示该方法带来更均衡的表征空间，提升长尾识别与迁移学习任务性能。
正文 · 原文
Abstract:Supervised contrastive learning and its balanced variants have demonstrated their effectiveness for representation learning under long-tailed distributions. However, although prior analyses suggest that their optimal representations may exhibit a regular simplex geometry, whether such a geometry is indeed exhibited remains insufficiently understood. In this work, we present a theoretical framework for these objectives that characterizes whether a regular simplex is formed at the optimum. By applying this to representative objectives, we show that these objectives fail to form a regular simplex at the optimum because they remain dependent on the number of samples per class. Motivated by this analysis, we introduce structural balancing that can be applied to contrastive objectives, and propose a fully balanced contrastive objective that provably forms a regular simplex geometry at its optimum. Both theoretical and empirical results demonstrate that the proposed method yields a more balanced representation space and improves performance on long-tailed recognition and transfer learning tasks.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.32039 [cs.LG] |
| (or arXiv:2610.32039v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32039 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kibok Lee [view email]
[v1]
Fri, 23 Oct 2026 01:30:14 UTC (692 KB)
来源：arXiv cs.AI · arxiv.org