MuSID-KD：面向语义 ID 生成式推荐的多粒度知识蒸馏方法

View PDF HTML (experimental)

Abstract:Generative recommendation with semantic IDs (SIDs) represents each item as a sequence of discrete tokens and autoregressively predicts the next item's SID. Despite strong recommendation performance, repeated decoding over multiple SID positions incurs substantial inference cost. Knowledge distillation transfers knowledge from a powerful generative teacher to a lightweight student, yet conventional objectives do not fully account for multi-token SID generation. We identify two limitations: item-level distribution alignment may not adequately preserve the teacher's ranking of candidate groups sharing SID prefixes, while uniform alignment across SID positions may affect final item ranking differently. We propose MuSID-KD, which combines Hierarchical Candidate Distribution Alignment (HCDA) and Ranking Impact Guided Levelwise Distillation (RIGD). HCDA aggregates item probabilities according to shared SID prefixes and aligns candidate distributions at successive levels, from candidate groups to individual items. RIGD evaluates the reduction in pairwise ranking surrogate loss when one student score contribution is replaced with the teacher's contribution, and allocates supervision accordingly. Experiments on real-world recommendation datasets using three lightweight student architectures demonstrate improved recommendation accuracy and better alignment of prefix marginal distributions and pairwise item rankings, without modifying the student's online inference architecture. Code is available at this https URL.

Comments:	12 pages, including appendix
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.35218 [cs.AI]
	(or arXiv:2610.35218v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.35218 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Fengshi Zhang [view email]
[v1] Mon, 26 Oct 2026 11:09:39 UTC (387 KB)