跳到正文
arXiv cs.AI· Xuefei Zhang, Ruoqi Wang, Lei Cui·· 2 小时前AI 评分32

角度条件蒸馏(ACD):解决分类器几何失配问题

Angular Conditional Distillation: Overcoming Classifier Geometry Mismatch

AI 导读

一篇 arXiv 论文提出角度条件蒸馏(ACD)方法,解决知识蒸馏中分类器几何失配导致的迁移不可靠问题。方法包含两个模块:角度坐标接口(ACI)用角度坐标重建学生分类器以抑制幅值与偏置影响,尺度不变条件蒸馏(SICD)标准化输出并只迁移条件性非目标关系。ACD 在实验中达到优于现有 KD 方法的 SOTA 表现,可作为通用即插即用替换,无需特征适配器且不增加推理成本,代码已开源。

正文 · 原文

View PDF HTML (experimental)

Abstract:Classifier outputs provide a fundamental and univer- sally available knowledge interface for knowledge distil- lation (KD). However, standard logit objectives entangle transferable semantic relations with architecture-dependent parameterization artifacts, causing a classifier geometry mismatch that makes direct knowledge transfer unreliable. To address this, we propose Angular Conditional Distilla- tion (ACD). First, the Angular Coordinate Interface (ACI) reconstructs the student classifier using angular coordi- nates, suppressing magnitude and bias effects to provide a geometry-compatible space. Then, Scale-Invariant Condi- tional Distillation (SICD) standardizes outputs and exclu- sively transfers conditional non-target relations, filtering out untransferable absolute confidence. Together, ACI de- fines where to transfer, while SICD defines what to transfer. Extensive experiments show that ACD achieves state-of-the- art performance over existing KD methods. Moreover, as a universal drop-in replacement, it consistently improves existing distillation pipelines while requiring no feature adapters or additional inference-time cost. Code is avail- able at this https URL.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23941 [cs.CV]
  (or arXiv:2611.23941v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23941

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Xuefei Zhang [view email]
[v1] Wed, 18 Nov 2026 06:28:17 UTC (2,345 KB)

来源:arXiv cs.AI · arxiv.org