arXiv cs.AI· Exauc\`e Luweh Adjim Ngarti (AIRSEA), \'Elise Arnaud (AIRSEA), L\'eo Nicoletti (Mines Saint-\'Etienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE), Victor Trappler (Mines Saint-\'Etienne MSE, LIMOS, FAYOL-ENSMSE, FAYOL-ENSMSE), Arthur Vidard (AIRSEA)·· 2 小时前AI 评分19
基于 Rényi 正则化的流变分推断方法用于贝叶斯校准
R\`enyi-Regularized Flow-Based Variational Inference for Bayesian Calibration
AI 导读
一项 arXiv 研究针对归一化流变分推断在多峰后验的贝叶斯模型校准中易丢失模式的问题，提出基于 Rényi 散度的正则项，促使近似后验覆盖先验支撑。在双峰孪生实验中，该正则化将模式坍缩比例从 50% 降至 25%，但会改变贝叶斯变分目标、带来近似精度权衡，并未给出模式坍缩的通用解法。
正文 · 原文
Abstract:Variational inference with normalizing flows provides a flexible posterior approximation for Bayesian model calibration, but struggles when the posterior has several modes separated by regions of low probability mass. We study this behavior in a two-parameter twin experiment and regularize the variational objective. Under Gaussian observation errors and a Gaussian prior, the corresponding minimization objective is the expected Tikhonov calibration objective minus the differential entropy of the approximating distribution. However, the expected Tikhonov term favors concentration near one of the modes, while entropy maximization does not guarantee multimodality. We introduce a regularization term based on Rényi divergence that encourages the approximate posterior to cover the prior support. In the connected bimodal case, a flow approximation represents both modes; in the case with well-separated modes, the flow and Gaussian mixture frequently miss one mode. The introduced regularization reduces the near-collapse fraction from 50% to 25%. While the regularization improves mode coverage in this experiment, it changes the Bayesian variational objective, introduces a trade-off in approximation accuracy, and does not provide a general solution to mode collapse.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2610.37169 [cs.AI] |
| (or arXiv:2610.37169v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37169 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Exauce LUWEH ADJIM NGARTI [view email] [via CCSD proxy]
[v1]
Tue, 27 Oct 2026 12:57:10 UTC (859 KB)
来源：arXiv cs.AI · arxiv.org