跳到正文
arXiv cs.AI· Ivan Karpukhin, Andrey Savchenko·· 2 小时前AI 评分25

AlignedHPO:基于代理目标的高效损失超参数优化框架

AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization

AI 导读

AlignedHPO 是一个利用梯度对齐和可微代理验证目标来调优可微损失超参数的框架,通过 Jacobian-vector products 降低超梯度计算的时间与内存开销,并以归一化对齐和梯度平滑解决梯度幅度歧义与随机不稳定问题。在广义 Focal Loss 和 ArcFace 的长尾分类与开集人脸识别实验中,相比隐式微分达到相当或更优性能,超梯度计算速度提升近 12 倍。

正文 · 原文

View PDF HTML (experimental)

Abstract:Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.22658 [cs.LG]
  (or arXiv:2611.22658v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.22658

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ivan Karpukhin [view email]
[v1] Tue, 17 Nov 2026 12:17:54 UTC (151 KB)

来源:arXiv cs.AI · arxiv.org