DiagTrace:一个用于临床决策的交互式基准
DiagTrace: An Interactive Benchmark for Clinical Decision Making
AI 导读
DiagTrace 是一个基于来源的临床决策基准,用于评估大语言模型能否通过交互式信息获取重构文档化的诊断轨迹。每个实例由 PubMed Central 病例报告构建,包含初始表现、事件库、临床过程和诊断参考,共覆盖八个专科的 510 个病例,其中 225 个用于评估四个 LLM 在临床过程对齐、交互质量与效率上的表现。结果显示模型在覆盖范围、与来源一致的排序和轮次使用上存在权衡。
正文 · 原文
Abstract:Large language models (LLMs) have shown strong performance on medical question answering and diagnostic reasoning benchmarks. However, existing evaluations typically provide all patient information upfront and assess only the final diagnosis or rationale. This setup ignores the need to acquire diagnostic evidence incrementally from incomplete information in real-world diagnosis. To bridge this gap, we introduce DiagTrace, a source-grounded benchmark for evaluating whether LLMs can reconstruct the documented diagnostic trajectory through interactive information seeking. Each benchmark instance is built from a PubMed Central case report and includes an initial presentation, an event bank, a clinical course, and diagnostic references. DiagTrace contains 510 cases across eight specialties, with 225 used to evaluate four LLMs on clinical-course alignment, interaction quality, and efficiency. Results reveal trade-offs in coverage, source-consistent ordering, and turn use.
| Comments: | Accepted to Findings of EACL 2027 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22009 [cs.CL] |
| (or arXiv:2611.22009v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22009 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yufan Wang [view email]
[v1]
Mon, 16 Nov 2026 23:42:28 UTC (166 KB)
来源:arXiv cs.AI · arxiv.org