AlignedHPO:基于代理目标的高效损失超参数优化框架
AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization
AI 导读
AlignedHPO 是一个利用梯度对齐和可微代理验证目标来调优可微损失超参数的框架,通过 Jacobian-vector products 降低超梯度计算的时间与内存开销,并以归一化对齐和梯度平滑解决梯度幅度歧义与随机不稳定问题。在广义 Focal Loss 和 ArcFace 的长尾分类与开集人脸识别实验中,相比隐式微分达到相当或更优性能,超梯度计算速度提升近 12 倍。
正文 · 原文
Abstract:Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22658 [cs.LG] |
| (or arXiv:2611.22658v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22658 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ivan Karpukhin [view email]
[v1]
Tue, 17 Nov 2026 12:17:54 UTC (151 KB)
来源:arXiv cs.AI · arxiv.org