BEiTScore++：基于 BEiT-3 与 Conformalized Quantile Regression 的免参考图像描述评估
BEiTScore++: Reference-free Image Captioning Evaluation with BEiT-3 and Conformalized Quantile Regression
AI 导读
研究者提出 BEiTScore++，为 BEiTScore 图像描述评估指标加入不确定性估计，可在单次前向传播中输出对齐分数和经保形校准的置信区间。在多个人工标注基准上，它保持了 BEiTScore 的排序相关性，区间宽度对预测误差的追踪在多数基准上优于 Monte Carlo dropout。不确定性会随图像退化及域外艺术品图像而增大。
正文 · 原文
Abstract:Reliable uncertainty estimation is important for deploying evaluation metrics in real-world settings, where both input ambiguity and domain shifts are common. In this work, we propose an uncertainty-aware extension of the BEiTScore image captioning metric, producing conformally calibrated confidence intervals alongside alignment scores. Our approach models predictive distributions over human judgments, enabling both accurate scoring and principled uncertainty quantification in a single forward pass. We evaluate our BEiTScore++ metric across multiple benchmarks with human-annotated ratings, showing that it preserves the ranking correlation of BEiTScore, while producing intervals whose width tracks the prediction error more closely than Monte Carlo dropout on most benchmarks. The estimated uncertainty also grows under image degradation and on out-of-domain artwork images, making BEiTScore++ a practical tool for uncertainty-aware evaluation of image-caption alignment.
| Comments: | Accepted to Findings of EMNLP 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.09505 [cs.CV] |
| (or arXiv:2611.09505v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09505 arXiv-issued DOI via DataCite |
|
| Journal reference: | Findings of the Association for Computational Linguistics: EMNLP 2026 |
| Related DOI: | https://doi.org/10.18653/2026.findings-emnlp.2160
DOI(s) linking to related resources |
Submission history
From: Gonçalo Emanuel Cavaco Gomes [view email]
[v1]
Fri, 6 Nov 2026 14:23:34 UTC (27,160 KB)
[v2]
Mon, 9 Nov 2026 21:51:00 UTC (27,160 KB)
来源：arXiv cs.AI · arxiv.org