轻量级概率基线 FBPR 可媲美 LLM：MedQA 临床诊断题研究
Counting Clues: A Lightweight Probabilistic Baseline Can Match an LLM
AI 导读
一项基于 MedQA 诊断题的研究提出 Frequency-Based Probabilistic Ranker（FBPR），用平滑朴素贝叶斯对概念-诊断共现统计打分。当共现数据来自 OLMo 预训练语料 Dolma 和 LLaMA 数据配方开源复现 RedPajama 时，FBPR 性能与对应 LLM 相当。LLM 推理与 FBPR 答对的题集重叠仅略高于随机，简单混合方法可带来显著增益。
正文 · 原文
Abstract:Large language models (LLMs) excel on multiple-choice clinical diagnosis benchmarks, yet it is unclear how much of this performance reflects underlying probabilistic reasoning. We study this through questions from MedQA that ask for the most likely diagnosis. We introduce Frequency-Based Probabilistic Ranker (FBPR), a lightweight method that scores options with a smoothed Naive Bayes over concept-diagnosis co-occurrence statistics from a large corpus. When co-occurrence statistics are sourced from Dolma, the pretraining corpus for OLMo, and RedPajama, an open reproduction of the LLaMA pretraining-data recipe, FBPR achieves comparable performance to the corresponding LLMs. Furthermore, direct LLM inference and FBPR exhibit complementary strengths, correctly answering largely different subsets of questions with overlap barely above chance. Combining them with a simple hybrid method yields a notable gain over either method alone. These findings show that explicit probabilistic baselines can account for a substantial portion of the benchmark performance on diagnosis questions, while serving as historically grounded reference points and complementary signals for hybridization.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2512.12868 [cs.CL] |
| (or arXiv:2512.12868v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2512.12868 arXiv-issued DOI via DataCite |
Submission history
From: Furong Jia [view email]
[v1]
Sun, 14 Dec 2025 23:00:10 UTC (1,196 KB)
[v2]
Tue, 27 Oct 2026 01:37:22 UTC (1,779 KB)
来源：arXiv cs.AI · arxiv.org