仅用 Image Transformer 配合后期像素精炼
Just Image Transformers with Late-Stage Pixel Refinement
AI 导读
研究者改进无 latent 扩散 transformer JiT：在 post-training 阶段加入卷积 pixel-pathway 层可显著提升图像生成质量，且该通路只需在去噪最后 20% 阶段激活。在 ImageNet@256 上 FID 从 2.34（corrected sampling）降至 1.78，无需额外损失函数。
正文 · 原文
Abstract:Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image ($x$-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last $20\%$ of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of $1.78$ (from $2.34$ with corrected sampling), without using additional losses.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13120 [cs.CV] |
| (or arXiv:2611.13120v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13120 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Denis Lukovnikov [view email]
[v1]
Mon, 9 Nov 2026 19:08:13 UTC (37,621 KB)
来源：arXiv cs.AI · arxiv.org