arXiv cs.AI· Daniel P. Jeong, Yewon Byun, John Giorgi, Peng Xu, Zachary C. Lipton, Michael Oberst·· 3 小时前AI 评分39
临床文档场景研究：利用用户编辑数据改进大语言模型能力的可行反馈方法
Actionable Feedback: Improving Model Capabilities from User Edit Data on Clinical Documentation
AI 导读
一项 arXiv 预印本研究提出从临床文档场景中的医生编辑数据里提取可行反馈来改进大语言模型。分析显示，在真实部署的 AI 临床文档产品数据中，超过 45% 被编辑的输出包含模型输入不支持的用户新增信息，直接微调会让模型学会幻觉。研究者在 MIMIC-III 出院小结生成任务上构建半合成编辑数据，并用过滤方法筛选有依据的编辑，在提升医生对齐指标的同时保持对输入的忠实性。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly being deployed in clinical documentation workflows, where clinicians review and edit their outputs before use. In principle, these clinician edits represent a scalable source of supervision for model training. In practice, not all edits represent actionable feedback: for instance, on data drawn from real-world deployment of an AI clinical documentation product, our review suggests that over 45% of edited outputs contain user-added information that is not directly supported by the inputs to the model. Naively fine-tuning on such edits risks teaching the model to hallucinate. In this paper, we develop a method to address this challenge. Given that real clinician edits on deployed documentation tools are not publicly available, we first develop a taxonomy of clinician-editing behaviors, informed by data from a deployed AI clinical documentation product. Then, we develop a method for generating semi-synthetic clinician-edit data, informed by our taxonomy, on a discharge-summary generation task based on MIMIC-III, a public dataset. Finally, we introduce simple filtering methods for identifying actionable edits, with accuracy evaluated by expert review. We show empirically, on our semi-synthetic dataset, that naive fine-tuning on raw edited output increases ungrounded content, while our approach of progressively filtering toward grounded edits enables improvement on clinician-alignment metrics while retaining faithfulness to the input context.
| Comments: | Preprint |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.12904 [cs.CL] |
| (or arXiv:2611.12904v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12904 arXiv-issued DOI via DataCite |
Submission history
From: Daniel Jeong [view email]
[v1]
Wed, 14 Oct 2026 19:54:40 UTC (656 KB)
来源：arXiv cs.AI · arxiv.org