社会世界模型评测缺口研究：审计 113 篇文献发现少有直接测量响应预测
The Social World Model Evaluation Gap
AI 导读
一项被 NeurIPS 2026 FMTS Workshop 接收的研究提出：现有社交世界模型评测很少直接检验系统对他人响应的预测能力。作者审计 1999 至 2026 年间的 113 篇文献，其中仅 9 篇主评测测量真实人对系统或实验者所选动作的响应，仅 2 篇同时比较显式响应预测与实际响应；对 25 篇语言模型论文的全文检查未发现符合这些条件的实验。研究还提出一个分离上述问题的单步谈判实验。
正文 · 原文
Abstract:A negotiation assistant can reach a good deal without accurately predicting the other person's response. We ask whether evaluations of social world models test that prediction directly. We audit 113 works published from 1999 to 2026. 9 primary evaluations measure a real person's response to an action selected by the system or experimenter. Only 2 also compare an explicit response forecast with that response. Their predictors use information available when the action is chosen, and the evaluated responses were excluded from fitting; both randomize the intervention. A separate full-text check of 25 language-model works finds no experiment meeting these conditions. The other evaluations test current-state inference, task success, or prediction from recorded actions. We propose a 1-step negotiation experiment that separates these questions. The findings describe the audited sample, not the prevalence of this evaluation gap across the field.
| Comments: | Accepted at the NeurIPS 2026 Workshop on Foundation Models for Temporal Systems: From Forecasting to World Modeling (FMTS). 4 pages of main text plus references and appendix. Code and data: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33014 [cs.CL] |
| (or arXiv:2610.33014v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33014 arXiv-issued DOI via DataCite |
Submission history
From: Denglin Jiang [view email]
[v1]
Thu, 8 Oct 2026 04:18:33 UTC (110 KB)
来源：arXiv cs.AI · arxiv.org