轨迹正则化分布蒸馏（TRD）实现单步生成
Trajectory-Regularized Distribution Distillation for One-Step Generation
AI 导读
研究者提出轨迹正则化分布蒸馏（TRD）方法，用于将流匹配模型蒸馏为一步生成模型。该方法通过单次学生前向传播预测动量混合策略，同时充当一步生成器、学生边缘速度估计器和受教师 ODE 轨迹监督的速度场，省去分布匹配中额外的可训练网络。
正文 · 原文
Abstract:Distilling flow matching models for one-step generation generally follows two paradigms. Trajectory alignment supervises the student with the teacher's probability flow ODE, but its performance degrades under very small sampling budgets. Distribution matching directly optimizes the generated distribution, but estimating its reverse Kullback-Leibler gradient typically requires a separate network to track the evolving student score. We propose trajectory-regularized distribution distillation (TRD), which combines distribution-level and trajectory-level supervision through a momentum-mixture policy predicted in a single student forward pass. This policy serves simultaneously as a one-step generator, an estimator of the student's marginal velocity that removes the need for a second trainable network in distribution matching, and a closed-form time-dependent velocity field supervised along the teacher ODE trajectory. Experiments on SANA-1.6B and Stable Diffusion 3.5 show that TRD outperforms representative consistency and distribution matching baselines in generation quality and training efficiency for both one-step and few-step generation, with a trade-off in sample diversity.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.12978 [cs.CV] |
| (or arXiv:2611.12978v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12978 arXiv-issued DOI via DataCite |
Submission history
From: An Zhao [view email]
[v1]
Sun, 8 Nov 2026 06:05:40 UTC (16,498 KB)
来源：arXiv cs.AI · arxiv.org