出院小结生成不只是摘要：FairSumEval 评估框架的现实检验
Discharge Summary Generation Is Not Just Summarization: A Reality Check for Evaluation
AI 导读
研究者提出证据感知的事实级评估框架 FairSumEval，用于检验出院小结生成是否只是摘要任务，区分有证据支持与无证据支持的内容差异。
正文 · 原文
Abstract:Discharge summaries combine a retrospective account of hospitalization with discharge-time decisions. Physician-written references therefore contain both facts supported by prior clinical notes and reference-only content, potentially reflecting clinical synthesis, handoffs, or discharge planning. Reference-only evaluation can conflate failure to recover available information with failure to reproduce content absent from the model input.
We introduce FairSumEval, an evidence-aware, fact-level framework that assesses whether each reference and generated fact is supported by the model's evidence and captured by the other summary. The resulting outcome map separates matches, omissions, and additions. Two complementary metrics, Soft IoU-S and Soft IoU-NS, measure reference-generation agreement for facts with and without identified evidence support.
On 200 MIMIC-III admissions, 18.8% of physician reference facts have identified support in the prior evidence, compared with 65.4% and 63.5% of facts generated by a general-purpose and a medically specialized model. Overlap is substantially higher for supported content (0.255 and 0.236) than for content without identified support (0.030 and 0.026). The two scores are weakly correlated across admissions (rho = 0.22 and 0.17), and conventional metrics do not consistently capture both dimensions. FairSumEval distinguishes evidence-grounded summarization from agreement on content without identified support, revealing what models omit and add in each category.
| Comments: | 20 pages, 8 figures. Radhika Dua and Frederick Wolff contributed equally. ML4H 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.14823 [cs.CL] |
| (or arXiv:2611.14823v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14823 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jan Frederick Wolff [view email]
[v1]
Tue, 10 Nov 2026 23:21:35 UTC (215 KB)
来源：arXiv cs.AI · arxiv.org