FinMultiHopQA：基于知识图谱的多跳金融问答基准发布
FinMultiHopQA: A KG-Grounded Multi-Hop Financial QA Benchmark for Evidence-Driven Reasoning
AI 导读
研究者发布 FinMultiHopQA 基准，包含 15,600 道定性与定量金融多跳问答题目，基于从 SEC 10-K 文件构建、含超 1750 万三元组的金融知识图谱 FinReflectKG。
正文 · 原文
Abstract:Multi-hop financial question answering often requires composing evidence distributed across filing sections, fiscal years, and peer companies. Prior financial QA benchmarks address parts of this problem individually, but none jointly support controlled multi-hop taxonomies, document topologies, motif-level reproducibility, and provenance-linked auditability at scale. We introduce FinMultiHopQA, a benchmark of 15,600 qualitative and quantitative questions, grounded in FinReflectKG, a temporally indexed financial knowledge graph derived from SEC 10-K filings with 17.5M+ triples. The pipeline instantiates statistically salient 2-3 hop typed KG motifs across intra-document, inter-year, and cross-company regimes. Beyond closing this gap, the same construction lets us hold each question's underlying evidence constant while varying only its presentation - embedding it in noisy surrounding context or changing its order - giving a controlled instrument to study a secondary question at scale: using six such presentations, we test whether reasoning LLMs' performance depends on evidence structure. Results confirm this: despite their reasoning capacity, judged correctness remains consistently lower under less-structured presentations of the same evidence - most sharply for quantitative cross-company reasoning and 3-hop branch motifs - indicating that reasoning alone does not fully offset how evidence is structured; KG-linked presentation also uses fewer input tokens by design. We release the benchmark, provenance annotations, model outputs, evaluation records, and supporting technical documentation.
| Comments: | 8 pages, 4 figures. Major revision of v1 with an expanded benchmark of 15,600 QA pairs, enhanced KG-driven question generation, additional evidence evaluation protocols, broader model evaluations, and expert quality validation |
| Subjects: | Computational Finance (q-fin.CP); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2510.02906 [q-fin.CP] |
| (or arXiv:2510.02906v2 [q-fin.CP] for this version) | |
| https://doi.org/10.48550/arXiv.2510.02906 arXiv-issued DOI via DataCite |
Submission history
From: Abhinav Arun [view email]
[v1]
Fri, 3 Oct 2025 11:19:31 UTC (2,227 KB)
[v2]
Wed, 4 Nov 2026 19:25:56 UTC (491 KB)
来源：arXiv cs.AI · arxiv.org