arXiv cs.AI· Aviv Orly (School of Physics and Astronomy, Tel Aviv University, Israel), Ori Shem-Ur (School of Physics and Astronomy, Tel Aviv University, Israel), Yaron Oz (School of Physics and Astronomy, Tel Aviv University, Israel)·· 3 小时前AI 评分44
一个温度统天下？大语言模型推理采样温度研究
One Temperature to Rule Them All?
AI 导读
特拉维夫大学研究团队在 pass@K 设置下分析多模型、多数据集上逐题成功概率随采样温度的变化，探讨温度多样性何时有益。研究发现更难的问题越少共享高效采样温度，但该趋势仅随难度呈对数变化；即使理论最优的多温度分配能提升表现，当温度按采样预算 K 调优后，其相对单温度基线的增益通常有限。
正文 · 原文
Authors:Aviv Orly (1), Ori Shem-Ur (1), Yaron Oz (1) ((1) School of Physics and Astronomy, Tel Aviv University, Israel)
Abstract:Inference-time sampling is a central mechanism for improving large language models, with temperature controlling the tradeoff between reliability and diversity. While recent work suggests that varying temperature across samples can improve multi-sample inference, simple fixed-temperature schemes remain the standard in practice. We investigate this discrepancy in the canonical pass@K setting, asking when temperature diversity is useful, and when a single tuned temperature suffices. To this end, we analyze per-question success probabilities across multiple models and datasets, tracking how they vary with sampling temperature. We identify several consistent properties of these profiles, including that harder questions are less likely to share efficient sampling temperatures, although this trend is only logarithmic in hardness. We then derive an efficient method for evaluating the theoretically optimal multi-temperature allocation scheme, and show that although such a scheme improves performance, its gains over single-temperature baselines are typically modest when temperature is tuned as a function of the sampling budget K. We explain this phenomenon through mathematical mechanisms that we call dulling effects, and show how they ensure the robustness of temperature choices across different question distributions. Our results clarify when multi-temperature sampling is useful, and how temperatures should ideally be chosen.
| Comments: | Accepted at NeurIPS 2026. 82 pages, 45 figures, 6 tables. Code: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.13077 [cs.LG] |
| (or arXiv:2611.13077v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13077 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aviv Orly [view email]
[v1]
Mon, 9 Nov 2026 19:00:01 UTC (18,550 KB)
来源：arXiv cs.AI · arxiv.org