跳到正文
arXiv cs.AI· Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein·· 2 小时前AI 评分59

斯坦福团队研究基于丰富自我报告的 AI 个体模拟:准确率提升至 72.8%,失真收窄

AI Simulations Grounded in Rich Self-Reports Increase Accuracy and Reduce Distortion

AI 导读

斯坦福团队测试丰富的自我报告数据能否提升 AI 个体模拟的准确性并减少样本级失真。团队用 AI 语音访谈器按 American Voices Project 协议对 1,052 名美国成年人进行两小时访谈,结合问卷与行为实验。

正文 · 原文

Authors:Joon Sung Park, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, Michael S. Bernstein

View PDF

Abstract:AI simulations of people built from demographic prompts often misjudge individuals and distort sample-level patterns. We test whether rich self-report data reduce these problems. We developed an AI voice interviewer and conducted two-hour interviews following the American Voices Project protocol with 1,052 U.S. adults, who also completed surveys and behavioral experiments. For each participant, we compared simulations prompted with demographic attributes, self-authored personas, survey responses, interview responses, or combined surveys and interviews. We evaluated accuracy on held-out responses, and measured eight statistical distortions at the sample level: biased averages, compressed variance, distorted response distributions, over-correlated items, attitudes collapsing onto one dimension, ideological sorting, amplified experimental effects, and hyper-rationality. On General Social Survey (GSS) items, demographic simulation led to 60.4% accuracy, whereas interview and survey simulation achieved 69.4% and 69.3%, and combining them reached 72.8%, compared with 79.5% for participants' two-week test-retest agreement. Surveys and interviews also improved prediction of personality traits, but not of incentivized economic choices. Combined interview and survey simulation closed a median of 53.5% of the eight distortion gaps between demographic simulation and participant responses. An open-weight model fine-tuned on other participants' self-reports and responses to the target questions achieved 75.2% GSS accuracy and closed a median of 69.3% of the distortion gap. Overall, both frontier and fine-tuned models reduced distortions without erasing them, but neither reduced hyper-rationality in economic games. These findings demonstrate that qualitative and quantitative self-reports can improve person-specific simulations and reduce statistical distortions.
Subjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Cite as: arXiv:2411.10109 [cs.AI]
  (or arXiv:2411.10109v4 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2411.10109

arXiv-issued DOI via DataCite

Submission history

From: Michael Bernstein [view email]
[v1] Fri, 15 Nov 2024 11:14:34 UTC (2,928 KB)
[v2] Wed, 22 Apr 2026 03:48:01 UTC (5,565 KB)
[v3] Sun, 28 Jun 2026 22:37:51 UTC (3,480 KB)
[v4] Mon, 16 Nov 2026 21:52:11 UTC (5,351 KB)

来源:arXiv cs.AI · arxiv.org