基于分位数曲线的免模型模拟器保真度评估方法(NeurIPS 2026)
Model-Free Assessment of Simulator Fidelity via Quantile Curves
AI 导读
一篇被 NeurIPS 2026 接收的 arXiv 论文提出了一种免模型方法来量化生成式 AI 模拟器与真实系统之间的 sim-to-real 差距。该方法为每个场景的潜在总体参数构建置信集,推导差距的稳健代理并估计其分位数函数,从而得到模拟器的分布级风险画像,支持 CVaR 等风险度量与跨模拟器比较。
正文 · 原文
Abstract:As generative AI models are increasingly used to simulate real-world systems, quantifying the ``sim-to-real'' gap is critical. For each input setting of interest -- which we call a \emph{scenario}, such as a survey question or operating condition -- the real and simulated systems are associated with unobserved latent population parameters, and their discrepancy varies across scenarios. A fundamental challenge is that, for any given scenario, this discrepancy cannot be observed directly, since both systems are accessible only through finite samples, often of heterogeneous sizes across scenarios. Standard predictive inference methods are therefore ill-suited, as they quantify uncertainty in observable outputs rather than latent population parameters. To address this, we construct confidence sets for these latent parameters and use them to derive a robust proxy for the sim-to-real discrepancy. We then estimate the quantile function of this proxy to obtain a distribution-level risk profile of the simulator, which supports a broad range of statistical summaries, including statistical inference for the real output distribution in a new scenario, the calculation of risk measures like Conditional Value-at-Risk (CVaR), and principled comparisons across simulators. Our method is model-agnostic and handles general output spaces, such as categorical survey responses and continuous multi-dimensional data. We demonstrate the practical utility of this method by evaluating the alignment of four major LLMs with human populations on the WorldValueBench dataset.
| Comments: | Accepted to NeurIPS 2026 |
| Subjects: | Methodology (stat.ME); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.05024 [stat.ME] |
| (or arXiv:2512.05024v4 [stat.ME] for this version) | |
| https://doi.org/10.48550/arXiv.2512.05024 arXiv-issued DOI via DataCite |
Submission history
From: Yu-Shiou Willy Lin [view email]
[v1]
Thu, 4 Dec 2025 17:39:51 UTC (2,122 KB)
[v2]
Sat, 31 Jan 2026 17:07:50 UTC (2,894 KB)
[v3]
Wed, 15 Apr 2026 22:25:30 UTC (2,672 KB)
[v4]
Fri, 13 Nov 2026 01:55:53 UTC (3,148 KB)
来源:arXiv cs.AI · arxiv.org