跳到正文
arXiv cs.AI· Jinrui Wu, Yun Peng·· 2 小时前AI 评分27

ProtoBridge-CLIP:观察掩码式病灶原型用于医学影像—报告表示学习

ProtoBridge-CLIP: Observation-Masked Finding Prototypes for Medical Image--Report Representation Learning

AI 导读

研究者提出 ProtoBridge-CLIP,将全局图文对称对齐与观察掩码的病灶监督结合,把报告中 affirmed、否定和未提及的概念区别处理,文本病灶原型与局部视觉 token 对齐。

正文 · 原文

View PDF HTML (experimental)

Abstract:Radiology reports distinguish affirmed findings, explicit negations, and unmentioned concepts, but conventional global image--report contrastive learning often compresses these different evidence states into a single study-level target. We introduce \model{}, which combines symmetric global image--report alignment with observation-masked finding supervision and a soft relation objective over affirmed report concepts. Text-derived finding prototypes are aligned with local visual tokens, while uncertain and unmentioned concepts are excluded from direct finding labels. We evaluate the framework on three CT benchmarks and five chest-radiograph benchmarks, together with component ablations, paired and clinical retrieval, localization, perturbation robustness, calibration, and per-finding analyses. \model{} achieves macro-AUROC values of 73.6%, 66.5\%, and 68.8% on CT-RATE, RAD-ChestCT, and INSPECT, and 80.2%, 91.4%, 78.5%, 86.5%, and 90.8% on ChestXray14, CheXpert, PadChest, RSNA, and SIIM-ACR, respectively. Relative to the matched global-only controls, the findings and relation objectives yield consistent gains across the evaluated datasets. These results demonstrate the practical value of treating observed and unobserved report concepts differently during medical image--text representation learning.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24002 [cs.CV]
  (or arXiv:2611.24002v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.24002

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Peng Yun [view email]
[v1] Wed, 18 Nov 2026 07:52:57 UTC (139 KB)

来源:arXiv cs.AI · arxiv.org