arXiv cs.AI· Tam\'as Janusko, Sven Kleinkop, Ricardo Bochnia, Gunnar Hempel, Steffen Tomschke, J\"urgen Anke, Maik Thiele·· 4 小时前AI 评分33
通过软提示微调让开源 LLM 评委对齐人类标注策略
Adapting LLM Judges to Observed Human Annotation Policy through Soft Prompt Tuning
AI 导读
一项 arXiv 预印本研究用软提示微调（soft prompt tuning）让开源 LLM 评委更贴近人类标注策略，在 FairBotBench 基准上针对偏差、毒性和个性化三个维度训练。混合式软提示调优——结合文本提示与学到的连续提示嵌入——持续提升与人类标注的一致性。Qwen 3.5 9B 适配后的评委在全部三个维度上超过托管参考评委，且比专有模型更易部署。
正文 · 原文
Abstract:Large Language Models (LLMs) are increasingly employed as automated judges for evaluating chatbot responses. While proprietary models can achieve high agreement with human annotators, they remain costly and difficult to deploy in practical settings. Lightweight open-source LLM judges provide an attractive alternative but typically exhibit lower agreement with human ratings. In this paper, we investigate whether soft prompt tuning can align open-source LLM judges more closely with human annotation policy. We train soft prompts on the FairBotBench benchmark and evaluate the adapted judges across the dimensions of bias, toxicity, and personalization. Our findings demonstrate that hybrid soft-prompt tuning, which combines textual prompts with learned continuous prompt embeddings, consistently improves agreement with human annotations over the corresponding untuned baselines. For Qwen 3.5 9B, the adapted judges surpass both hosted reference judges across all three evaluation dimensions.
| Comments: | CONVERSATIONS 2026, International Symposium on Chatbots and Human-Centred AI (Preprint) |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2611.07126 [cs.CL] |
| (or arXiv:2611.07126v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07126 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Maik Thiele [view email]
[v1]
Wed, 4 Nov 2026 20:48:34 UTC (383 KB)
来源：arXiv cs.AI · arxiv.org