跳到正文
arXiv cs.AI· Marzieh Karimi, Mehran Safayani, Alireza Basiri·· 2 小时前AI 评分16

TUSI:基于引用溯源的波斯法律 RAG 图结构基准

TUSI: A Citation-Grounded Graph-Based Benchmark

AI 导读

研究者推出 TUSI-Bench,一个面向波斯语法律 RAG 系统评测的结构化基准,其多层法律知识框架涵盖 7,261 条法律条文,并构建连接条文与概念的图结构。基准包含两个任务:TUSI-QA(1,999 道要求带明确法律引用溯源的开放式问答题)和 TUSI-Bar(约 500 道来自伊朗律师资格考试的四选项选择题)。基于图结构的检索基线使检索召回率较标准基线最多提升 8 个百分点。

正文 · 原文

View PDF HTML (experimental)

Abstract:Evaluating retrieval-augmented generation (RAG) systems in Persian law remains challenging because benchmarks lack realism, grounding, and coverage of legal reasoning. Existing resources are either synthetically generated from statutory texts, lacking natural information needs, or derived from informal legal question--answer sources without explicit linkage to authoritative legal documents, making them unsuitable for rigorous evaluation of grounded legal reasoning. To address this gap, we introduce TUSI-Bench, a structured Persian legal benchmark designed for the evaluation of RAG-based systems. TUSI-Bench is built upon a multi-layered legal knowledge framework containing 7,261 legal articles and integrating codified statutes, legal commentaries, doctrinal writings, and Supreme Court unification rulings. We also construct a legal knowledge graph linking articles and concepts for structure-aware retrieval. Based on this foundation, TUSI-Bench provides two complementary evaluation tasks: (i) TUSI-QA, an open-ended question answering benchmark containing 1,999 questions that requires grounded responses with explicit legal citations, and (ii) TUSI-Bar, a multiple-choice benchmark containing approximately 500 four-option questions derived from Iranian Bar examination questions. Furthermore, we propose a graph-based retrieval baseline that leverages relational structure within the legal knowledge graph. Experimental results demonstrate that incorporating graph-based relations improves retrieval recall by up to 8 percentage points compared to standard baselines, highlighting the importance of structured legal representations for RAG systems. The proposed benchmark provides a standardized resource for evaluating grounded legal reasoning in Persian and enables future research on structured legal intelligence.
Comments: 28 pages, 11 figures, including appendix
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21823 [cs.CL]
  (or arXiv:2611.21823v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21823

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Marzieh Karimi [view email]
[v1] Mon, 16 Nov 2026 19:38:41 UTC (1,145 KB)

来源:arXiv cs.AI · arxiv.org