GraphDancer：用课程强化学习训练大语言模型在图上探索与推理
GraphDancer: Training LLMs to Explore and Reason over Graphs via Curriculum Reinforcement Learning
AI 导读
GraphDancer 是一个强化学习框架，通过推理与函数调用交替执行训练大语言模型在异构知识图上导航与多跳推理，并引入按问题结构难度调度的图感知课程采样。
正文 · 原文
Abstract:Large language models (LLMs) increasingly rely on external knowledge to improve factuality, yet many knowledge sources on the Web, such as scholarly indexes and product catalogs, are organized as heterogeneous graphs. Reasoning over such graph-structured knowledge poses two key challenges: (1) navigating structured, schema-defined relations requires precise function calls beyond similarity-based retrieval, and (2) answering complex questions often demands multi-hop evidence aggregation through iterative information seeking. We study how LLMs can learn these interaction capabilities and transfer them to unseen domains. We propose GraphDancer, a reinforcement learning (RL) framework that teaches LLMs to navigate graphs by interleaving reasoning and function execution. To improve RL for moderate-sized LLMs, we introduce a graph-aware curriculum that schedules training by the structural difficulty of questions using an easy-to-hard biased sampler. We evaluate GraphDancer on the multi-domain GRBench benchmark, training only on Academic and Biomedical graphs. Testing covers held-out questions from these domains as well as unseen E-commerce, Literature, and Legal graphs. With either Qwen2.5-3B-Instruct or Qwen3-14B as the backbone, GraphDancer outperforms graph-agent baselines using the same backbone across all evaluation domains. These results indicate that the learned graph exploration and reasoning capabilities transfer across domains. On average across unseen domains, GraphDancer with a 14B backbone also outperforms baselines powered by the closed-source models GPT-4.1-mini and GPT-5-nano. Trajectory analyses further show that GraphDancer retrieves answer-bearing evidence more often and achieves higher accuracy when such evidence is found, while substantially reducing invalid-protocol outcomes.
| Comments: | 14 pages. Code: this https URL. Project website: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.02518 [cs.LG] |
| (or arXiv:2602.02518v3 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.02518 arXiv-issued DOI via DataCite |
Submission history
From: Yuyang Bai [view email]
[v1]
Sat, 24 Jan 2026 02:44:49 UTC (143 KB)
[v2]
Tue, 26 May 2026 02:25:19 UTC (177 KB)
[v3]
Sun, 25 Oct 2026 14:54:07 UTC (169 KB)
来源：arXiv cs.AI · arxiv.org