基于冻结语言模型的医疗选择题 System 1 决策模型研究
A System 1 Decision Model for Medical Multiple-Choice Questions Built on Frozen Language Models
AI 导读
一项 arXiv 研究在冻结的 Qwen3.5-0.8B、Granite-4.2-3B 和 Qwen3-8B 上仅训练两个投影头和一个温度参数,构建 System 1 单次通过模型作答 MedMCQA 医疗选择题。
正文 · 原文
Abstract:A System 1 model answers a multiple-choice question in a single pass. A frozen language model turns the question and each option into vectors, two small, trained heads place them in a shared space, and the closest option wins. The design has been shown on agent decisions, but we are not aware of a study of how far it goes on exam questions that need medical knowledge, where the right option is often no more similar to the question than the wrong ones. We rebuilt the recipe from scratch for MedMCQA and trained only two projection heads and one temperature on top of frozen Qwen3.5-0.8B, Granite-4.2-3B and Qwen3-8B, using a contrastive loss with in-batch and hard negatives and a mask for repeated answers. With plain last-token vectors, both small encoders answer about one question in three correctly (33.1% and 33.0% on 3,842 held-out questions, where chance is 25.0%). Writing each text inside a one-word-summary prompt and lowering the learning rate of the heads raised this to 41.7% and 42.5% for the two small encoders and to 46.0% +-0.2 (three seeds) for Qwen3-8B. Shuffling the questions against the options returns accuracy to chance, so the heads do use the question. Flan-PaLM and Med-PaLM 2 report 57.6% and 72.3% on MedMCQA under a different protocol, so ours is a cheap, fast baseline and not a competitive system. We also say what our evidence cannot show: validation gains were about 1.7 to 2 times the test gains, and we could not separate the effect of reading several encoder layers from the other changes.
| Comments: | 12 pages with 7 tabular explanations |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23832 [cs.AI] |
| (or arXiv:2611.23832v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23832 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pavan Kumar Mantha [view email]
[v1]
Wed, 18 Nov 2026 04:38:49 UTC (18 KB)
来源:arXiv cs.AI · arxiv.org