arXiv cs.AI· Zhichao Yang, Sepehr Janghorbani, Dongxu Zhang, Jun Han, Qian Qian, Andrew Ressler II, Gregory D. Lyng, Sanjit Singh Batra, Robert E. Tillman·· 4 小时前AI 评分27
Health-SCORE：面向健康领域 LLM 的可扩展评分标准框架
Health-SCORE: Towards Scalable Rubrics for Improving Health-LLMs
AI 导读
Health-SCORE 是一个可泛化、可扩展的基于评分标准的医疗 LLM 训练与评估框架，大幅降低 rubric 开发成本而不牺牲性能。它既可作为结构化奖励信号指导带安全感知监督的强化学习，也可直接嵌入提示词通过 in-context learning 提升回复质量。在开放式医疗任务上，其评估质量与人造 rubric 相当，发表于 EMNLP 2026 Findings。
正文 · 原文
Abstract:Rubrics are essential for evaluating open-ended LLM responses, especially in safety-critical domains such as healthcare. However, creating high-quality and domain-specific rubrics typically requires significant human expertise time and development cost, making rubric-based evaluation and training difficult to scale. In this work, we introduce Health-SCORE, a generalizable and scalable rubric-based training and evaluation framework that substantially reduces rubric development costs without sacrificing performance. We show that Health-SCORE provides two practical benefits beyond standalone evaluation: it can be used as a structured reward signal to guide reinforcement learning with safety-aware supervision, and it can be incorporated directly into prompts to improve response quality through in-context learning. Across open-ended healthcare tasks, Health-SCORE achieves evaluation quality comparable to human-created rubrics while significantly lowering development effort, making rubric-based evaluation and training more scalable.
| Comments: | Published at EMNLP 2026 - Findings |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2601.18706 [cs.AI] |
| (or arXiv:2601.18706v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2601.18706 arXiv-issued DOI via DataCite |
Submission history
From: Sepehr Janghorbani [view email]
[v1]
Mon, 26 Jan 2026 17:34:10 UTC (5,396 KB)
[v2]
Sun, 25 Oct 2026 01:24:23 UTC (7,497 KB)
来源：arXiv cs.AI · arxiv.org