Semantic Prism:结合显式语义-图像界面的生成式道路场景分割框架
Semantic Prism: Generative Segmentation with an Explicit Semantic-Image Interface
AI 导读
Semantic Prism 框架将显式语义-图像接口与层级细化相结合,用于道路场景生成式分割,由 diffusion-distilled 骨干改造的生成器每次 crop 仅需一次去噪更新。
正文 · 原文
Abstract:Conditional image generators can render segmentation maps as semantic RGB images, but color drift and boundary mixing can undermine direct decoding. We present Semantic Prism, a framework that combines an explicit semantic-image interface with hierarchical refinement for road-scene segmentation. A generator adapted from a diffusion-distilled backbone produces a semantic RGB image with a single denoising update per crop. Distances to a fixed class-color codebook define interface logits and pixelwise class probabilities. Hierarchical Generator Evidence Alignment (HGEA) combines spatially aligned multilevel features from the VAE encoder of the generator with the input image, generated semantic image, and interface probabilities to predict additive corrections to these logits. Contextual Interface Hierarchy Disagreement (C-IHD) combines pointwise uncertainty, local context, and interface-refinement disagreement in a fixed score for ranking pixel errors, without an auxiliary predictor or an additional network pass. On the 500-image Cityscapes validation set, Semantic Prism achieves 72.07% mean mIoU, improving on direct-interface decoding by 11.39 percentage points. Extensive capacity-matched ablations show higher mean mIoU for multilevel refinement than for flat and single-level alternatives. The Cityscapes-trained model transfers to ACDC Dataset, achieving 46.89% mIoU without target-domain adaptation. Beyond Cityscapes, on BDD100K dataset, a separately trained model achieves 62.22% mIoU on its full 1,000-image validation set. Across the 3 evaluated datasets, C-IHD improves pixel-error area under the AUPR over uncertainty derived from maximum softmax probability, with segmentation predictions held fixed for each comparison. In particular, on ACDC, AUPR increases from 0.6580 to 0.7557, demonstrating that Semantic Prism presents more effective error ranking under adverse-condition domain shift.
| Comments: | 16 pages, 4 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Image and Video Processing (eess.IV) |
| Cite as: | arXiv:2608.11537 [cs.CV] |
| (or arXiv:2608.11537v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2608.11537 arXiv-issued DOI via DataCite |
Submission history
From: Yongqi Dong [view email]
[v1]
Wed, 12 Aug 2026 01:00:04 UTC (6,564 KB)
[v2]
Tue, 17 Nov 2026 17:08:08 UTC (12,408 KB)
来源:arXiv cs.AI · arxiv.org