基于 Prior-fitted Diffusion Transformer 的 Thompson Sampling 研究获 NeurIPS 2026 接收
Thompson Sampling using Prior-fitted Diffusion Transformers
AI 导读
一篇 NeurIPS 2026 论文提出 prior-fitted diffusion transformer，通过先验拟合网络范式预训练，在推理时于上下文内执行 Thompson Sampling，突破了以往仅限高斯过程先验的局限。
正文 · 原文
Abstract:Thompson sampling is natural randomized strategy for optimizing unknown functions in a black-box manner. In continuous global optimization, Thompson sampling has largely been restricted to Gaussian process priors, as these are essentially the only class for which computing the posterior distribution has been tractable in practice. In this work, we introduce \emph{prior-fitted diffusion transformers}, which are diffusion models that perform Thompson sampling in-context at inference time, after being pre-trained using the prior-fitted network paradigm. To apply these models, we develop a set of practical non-Gaussian priors for general black-box functions---including those that have characteristics that cannot be achieved under Gaussianity, such as priors over unimodal functions. We also quantify the tradeoff between model size and the number of pre-training iterations, and show that these models follow scaling laws akin to those observed in transformers defined over other modalities. Across a set of global optimization benchmarks, we show that prior-fitted diffusion transformers with non-Gaussian priors can achieve improved performance compared to traditional pipelines built on Gaussian processes.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2611.14656 [cs.LG] |
| (or arXiv:2611.14656v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14656 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | Advances in Neural Information Processing Systems, 2026 |
Submission history
From: Alexander Terenin [view email]
[v1]
Tue, 10 Nov 2026 19:07:33 UTC (408 KB)
来源：arXiv cs.AI · arxiv.org