arXiv cs.AI· Sarah Schr\"oder, Thekla Morgenroth, Ulrike Kuhl, Valerie Vaquet, Benjamin Paa{\ss}en·· 4 小时前AI 评分42
LLM 能否模拟人类认知的极限探索：以道德判断为例的研究警示
Exploring the Limits of Large Language Models as a Reflection of Human Cognition: An Illustration in the Context of Moral Judgement
AI 导读
一项 arXiv 研究警示，不宜把 ChatGPT 等大语言模型直接当作心理学研究中的被试模拟器。实验对既有研究中使用的 30 个道德场景措辞稍作改动，人类与大语言模型评分的相似度就显著下降，且下降幅度因模型而异。作者认为 LLM 在新刺激上的行为可能与人类被试不同，建议研究者针对具体实验先验证模型与人类回答的相似性。
正文 · 原文
Abstract:Large Language Models (LLMs), such as ChatGPT, are increasingly used in research, ranging from simple writing assistance to complex data annotation tasks. Recently, some research has suggested that LLMs are able to reproduce human responses to psychological questions and may, therefore, be used as simulators of human participants in some psychological research. We caution against this approach. Based on machine learning research, we argue that LLMs are likely to behave differently on novel stimuli compared to human study participants. We exemplify this in an experiment regarding moral judgments, demonstrating that slight changes to the wording of 30 moral scenarios used in prior work significantly reduce the similarity between ratings made by humans and LLMs, although the magnitude of this reduction varies across models. Therefore, we advise researchers to validate the similarity between human and LLM responses for their specific experiment.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.06950 [cs.AI] |
| (or arXiv:2508.06950v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2508.06950 arXiv-issued DOI via DataCite |
|
| Related DOI: | https://doi.org/10.1037/tmb0000210
DOI(s) linking to related resources |
Submission history
From: Benjamin Paassen [view email]
[v1]
Sat, 9 Aug 2025 11:56:59 UTC (2,197 KB)
[v2]
Tue, 12 Aug 2025 07:58:10 UTC (89 KB)
[v3]
Wed, 13 Aug 2025 14:59:57 UTC (89 KB)
[v4]
Sun, 25 Oct 2026 13:55:37 UTC (556 KB)
来源：arXiv cs.AI · arxiv.org