DTO：面向文生图扩散模型推理时对齐的直接轨迹优化方法
Direct Trajectory Optimization for Inference-Time Alignment of Text-to-Image Diffusion Models
AI 导读
研究者提出 Direct Trajectory Optimization（DTO），一种免训练的文生图扩散模型对齐框架，直接优化采样轨迹，将中间去噪分数和采样噪声作为优化变量，无需对去噪网络做反向传播。
正文 · 原文
Abstract:Alignment of text-to-image diffusion models with reward functions is a key challenge in controllable generation. Existing approaches typically rely on optimizing model inputs or fine-tuning denoising networks, which requires computationally expensive backpropagation through large denoising models. In this work, we propose Direct Trajectory Optimization (DTO), a training-free framework that performs alignment by directly optimizing the sampling trajectory of diffusion models. Specifically, DTO treats intermediate denoising scores and sampled noise as the optimization variables, eliminating the need for gradients through the denoising network. To mitigate the out-of-distribution problem, we introduce Reward-Optimized Score Guidance, which interpolates between the optimized scores and the pretrained model predictions to stabilize sampling, while providing a provable theoretical guarantee of reward improvement under constrained distributions. Empirically, DTO improves alignment performance across three diffusion models on two benchmarks, outperforming existing inference-time alignment methods while maintaining high efficiency. These results suggest that trajectory-level optimization is a promising direction for efficient and effective diffusion model alignment.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.11821 [cs.CV] |
| (or arXiv:2611.11821v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.11821 arXiv-issued DOI via DataCite |
Submission history
From: Huixuan Zhang [view email]
[v1]
Mon, 9 Nov 2026 03:29:50 UTC (12,609 KB)
[v2]
Tue, 10 Nov 2026 05:52:03 UTC (12,609 KB)
来源：arXiv cs.AI · arxiv.org