基于 PAC-Bayes 风险的嵌套因果老虎机认证策略优化方法
Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk
AI 导读
该 arXiv 论文提出嵌套上下文因果老虎机(NCCB)问题类与 Nested Causal Thompson Sampling(NCTS)算法,用分层 SCM 建模战略决策对后续战术选择的因果影响。
正文 · 原文
Abstract:Critical sequential decisions are rarely single-timescale: a strategic decision causally shapes the context in which every subsequent tactical choice is made; standard bandit and reinforcement-learning theory does not capture this causal coupling between timescales. We formalise the problem class as Nested Contextual Causal Bandits (NCCBs), a hierarchical SCM where each level's action sets the next level's context distribution, and propose Nested Causal Thompson Sampling (NCTS), which draws one mechanism-factorised belief per episode and acts recursively under it. Our main theoretical result is a causal PAC-Bayesian excess-risk bound that certifies any candidate deployment policy from historic data alone, off-policy and anytime, answering the deployment question: can we trust this agent here, and at what risk? Experiments on a hierarchical SCM show that, against a matched RFF-GP joint regression on the same function class, the factorised SCM-mechanism posterior transfers significantly better zero-shot under exogenous distribution shifts, the recursive meta-to-inner commit significantly dominates the joint-commit alternative in distribution, and the certificate significantly contracts as offline data accumulates. Combining these results, we establish progressive certified handover, a safe-deployment method: each timescale flips from a legacy controller to NCTS when gains can be certified, independently of the others.
| Comments: | Accepted at NeurIPS 2026 (Main Track) |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2605.29788 [cs.AI] |
| (or arXiv:2605.29788v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.29788 arXiv-issued DOI via DataCite |
Submission history
From: Tim Woydt [view email]
[v1]
Thu, 28 May 2026 11:39:51 UTC (163 KB)
[v2]
Fri, 13 Nov 2026 13:10:31 UTC (196 KB)
来源:arXiv cs.AI · arxiv.org