arXiv cs.AI· Zehao Gao, Jie Feng, Wenda Zhao, Junpeng Zhang, Ronghua Shang, Weisheng Dong·· 2 小时前AI 评分34
EmergeFuse:面向开放词汇语义分割的类相对红外-可见光融合框架
Fuse to Recognize the Unseen: Class-Relative Infrared-Visible Fusion for Open-Vocabulary Semantic Segmentation
AI 导读
arXiv 论文提出 EmergeFuse,一种面向开放词汇语义分割的红外-可见光融合框架,可选择性将热证据转化为未见类别的语义校正。该方法通过类相对线索选择和文本对比图像校正,避免红外信息破坏 CLIP 的 RGB-语言对齐。在三个基准上,未见类 mIoU 最高提升 4.755%,严格互补转化率提升 15.730%,同时保持有竞争力的融合质量。
正文 · 原文
Abstract:Visible-infrared image fusion leverages complementary thermal cues to enhance visual perception, particularly under challenging illumination. However, such perceptual complementarity does not readily translate into semantic gains in open-vocabulary semantic segmentation (OVSS), where CLIP-based models are predominantly pretrained on RGB imagery. Directly introducing IR information may instead disrupt RGB-language alignment, turning promising thermal cues into semantic interference that undermines unseen-class recognition. Our pixel-level RGB-IR semantic analysis reveals that class-relevant thermal cues can correct certain RGB misclassifications while preserving reliable RGB predictions, motivating RGB-anchored, class-relative thermal cue selection. Based on this insight, we propose EmergeFuse, an infrared-visible fusion framework for OVSS that selectively converts complementary thermal evidence into unseen-class semantic corrections. Specifically, class-relative cue selection evaluates RGB-IR semantic responses and calibrated thermal evidence relative to RGB predictions, generating class-relative correction requests through competitive selection. Text-contrast image rectification then translates these selected requests into semantic rectification directions through text contrasts, guiding evidence-gated feature updates to realize complementary thermal evidence for unseen-class recognition. Experiments on three benchmarks demonstrate gains of up to 4.755% in unseen-class mIoU and 15.730% in strict complementary conversion rate over existing fusion methods, while maintaining competitive fusion quality.
| Comments: | 11 pages, 6 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24316 [cs.CV] |
| (or arXiv:2611.24316v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24316 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zehao Gao [view email]
[v1]
Wed, 18 Nov 2026 12:22:54 UTC (11,078 KB)
来源:arXiv cs.AI · arxiv.org