NxN e-valuation：基于 Conformal CRT 零假设的语言模型假设认证方法
NxN E-valuation: Hypothesis Certification via a Conformal CRT Null
AI 导读
一项 arXiv 论文提出 NxN e-valuation，为语言模型生成的假设提供自动化认证：以同一 scope 内的单元互为零假设，无需为每个假设写重采样模型，并用 betting e-value 在有限样本内控制假认证概率。
正文 · 原文
Abstract:Language models now propose hypotheses faster than anyone can check them, so the step that certifies them must need nothing from the proposer but the hypothesis. We state three requirements for such a step -- per-hypothesis automatable, attributable and diagnostic -- and introduce NxN e-valuation, which meets them. A hypothesis names a scope, the units it is about, and gives each of them a restriction that scores its outcome. The units of one scope serve as nulls for one another, so no resampling model is written per hypothesis; the algorithm certifies a stable, scope-level association and reports whether the scope, the restriction, or which unit receives which restriction carries the prediction. For independently drawn units, betting e-values bound the chance of a false certificate for each hypothesis in finite samples, under stated nulls. A betting lower bound on the share of units that carry the association gates every certificate, and an asymptotically valid equivalence test calls the restrictions shared only when it rejects differences between them above a declared noise ceiling. We test it on planted data, on randomized intervention data, and on real purchase data with 285 hypotheses that language models wrote without knowing our method or our classes, labeled by human experts. Meeting all three requirements costs no accuracy: NxN makes 692 correct decisions in 693 with one false certificate, against 659 with four for a composite of CRT tests that returns our verdicts but needs a resampling model per hypothesis, and 224 with 468 for held-out validation. Removing any of our statistics merges a verdict that certifies with one that rejects. The certificates concern predictive association, not causality.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.06621 [cs.AI] |
| (or arXiv:2608.06621v5 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.06621 arXiv-issued DOI via DataCite |
Submission history
From: Bin Wang [view email]
[v1]
Thu, 6 Aug 2026 22:14:53 UTC (50 KB)
[v2]
Sat, 29 Aug 2026 10:49:17 UTC (98 KB)
[v3]
Thu, 3 Sep 2026 22:12:40 UTC (99 KB)
[v4]
Mon, 5 Oct 2026 08:47:09 UTC (124 KB)
[v5]
Mon, 26 Oct 2026 19:19:30 UTC (121 KB)
来源：arXiv cs.AI · arxiv.org