RaRe：面向开放词汇场景图生成的推理感知关系表示学习框架
Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation
AI 导读
RaRe 是一个推理感知的关系表示学习框架，将 MLLM 生成的思维链（CoT）论据转化为结构化关系嵌入，并直接优化该表示空间以提升关系可区分性。该方法先通过自监督对比学习对齐论据嵌入与关系描述文本，再用面向特异性的强化学习目标优化生成。在 VG150 和 PSG 上的实验显示类别均衡识别优于现有基线，代码已开源。
正文 · 原文
Abstract:Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: this https URL.
| Comments: | Accepted by NeurIPS 2026, codes are available at project page this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13496 [cs.CV] |
| (or arXiv:2611.13496v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13496 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Longtian Qiu [view email]
[v1]
Tue, 10 Nov 2026 03:08:22 UTC (7,652 KB)
来源：arXiv cs.AI · arxiv.org