评估 Agentic RAG 中的 Conformal 事实性：自适应检索与选择性回答

View PDF HTML (experimental)

Abstract:Retrieval-augmented generation (RAG) can improve factual grounding without guaranteeing that every generated claim is supported. Previous work has studied claim-level conformal factuality in single-stage RAG and, more recently, multi-hop RAG. We examine its behavior in agentic RAG, where a language-model policy adaptively issues follow-up retrieval queries or terminates retrieval before response generation. We retain the retrieval-based relevance-scoring formulation and post-generation conformal filtering and evaluate six model-dataset configurations: GPT-6 Luna and Llama 3.1 8B on HotpotQA, Natural Questions, and TriviaQA. Each configuration uses 60 fixed questions, 100 seeded 30/30 calibration-test partitions, five conformal target levels, and a no-conformal-prediction baseline. Across the six configurations, higher target levels are accompanied by higher observed full-support rates and reduced answer availability. At the 95% target, non-empty response coverage ranges from 17.23% to 60.17%, claim retention from 10.72% to 43.16%, and full support among non-empty outputs from 89.14% to 95.56%. We formulate the adaptive retrieval trajectory explicitly and characterize the reliability-informativeness trade-off when conformal factuality is applied after agent termination.

Comments:	10 pages, 1 figure, 5 tables. Code: this https URL
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.34872 [cs.CL]
	(or arXiv:2610.34872v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.34872 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Muhammad Aimal Rehman [view email]
[v1] Mon, 26 Oct 2026 05:37:03 UTC (48 KB)