arXiv cs.AI· Hoang Pham Cong, Long D. Nguyen, Ho Viet Duc Luong, Nguyen Tran Minh Nhat, Binh P. Nguyen, Huynh Thi Thanh Binh·· 1 小时前AI 评分39
冻结多模态模型在模态缺失下的风险可控修正方法 GUARD
Risk-Controlled Correction for Frozen Multimodal Models under Missing Modalities
AI 导读
arXiv 论文提出 GUARD(Gated Utility-Aware Recalibration for Deployment),针对权重已冻结的多模态模型在模态缺失时的预测误差,估计可修正部分并仅在有益时施加修正。
正文 · 原文
Abstract:A deployed multimodal predictor may face missing modalities after its weights can no longer be changed. This raises a question that training-time repair does not answer: what part of the resulting error is still correctable from the information that remains? We show that this part is characterized by the Bayes gap, the difference between the model's prediction and the Bayes prediction given the observed modalities. For cross-entropy and squared error, excess risk measures this gap in the geometry of the loss; ambiguity in the observed modalities cannot be removed. Thus, a badly degraded prediction is not necessarily a correctable one. Estimating the Bayes prediction normally requires labels. However, when a model observing more modalities is conditionally correct, averaging its outputs conditional on the remaining modalities recovers this target without retrieval labels; with the same fixed blend and neighbors, it is no worse in expected cross-entropy or Brier risk than averaging hard labels. Some such assumption is unavoidable: otherwise the target is not identifiable from unlabeled data alone. GUARD (Gated Utility-Aware Recalibration for Deployment) leaves the deployed predictor unchanged. It estimates a candidate correction by retrieval, learns from deployment-matched labeled data when correction is likely to help, and calibrates a threshold on a held-out split by conformal risk control. Under exchangeability, the marginal probability that GUARD applies a correction whose loss increase exceeds a prescribed tolerance is bounded by a preset level, regardless of the learned score. In controlled experiments, correction gain follows headroom rather than raw error; across nine frozen model-dataset pairs spanning six domains, GUARD retains most of blanket correction's gain within harm budgets that blanket correction often exceeds.
| Comments: | Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2611.23886 [cs.AI] |
| (or arXiv:2611.23886v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23886 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hoang Pham Cong [view email]
[v1]
Wed, 18 Nov 2026 05:39:35 UTC (438 KB)
来源:arXiv cs.AI · arxiv.org