跳到正文
arXiv cs.AI· Himaghna Bhattacharjee, Yasha Sheynin, Shivam Singhal, Arjun Subramonian, Ankur Samanta, Julian Michael, Smitha Milli, Maximilian Nickel, Claudia Shi·· 56 分钟前AI 评分49

从偏好数据到个性化:追踪大语言模型中的谄媚行为

From Preference Data to Personalization: Tracing Sycophancy in Large Language Models

AI 导读

一项研究将 LLM 谄媚定义为不当认同,提出 validity-aware 谄媚评分器和含 3,298 条标注探针的 SycoDrift 评测集,并分析人类偏好数据、评测 judge 规范和上下文个性化三个认同压力来源。

正文 · 原文

View PDF HTML (experimental)

Abstract:Sycophancy in large language models is often measured as agreement with the user, yet agreement can be appropriate when the user's claim is valid, their preference is genuine, or their experience warrants acknowledgment. We study sycophancy as inappropriate agreement: validating a claim, preference, or self-assessment when the context calls for correction, qualification, or resistance. We introduce a validity-aware sycophancy grader and SycoDrift, a persona-targeted evaluation set of 3,298 validity-annotated probes with controlled annotations for user openness and stakes. We use this framework to analyze three sources of agreement pressure: human preference data, automated judge specification, and in-context personalization. First, preference for validating responses in human datasets (Chatbot Arena, HH-RLHF, Community Alignment) is weak and concentrated where agreement is often appropriate. Second, in a selected-value, single-model experiment, emphasized judge rubrics produce consistent but modest increases relative to paired suppressed rubrics, demonstrating a plausible mechanism for training-time amplification. Third, in off-policy evaluations across 14 models, inference-time personalization increases inappropriate agreement, with the largest effect from matched user profiles placed directly before the query.
Comments: 66 pages, 19 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21650 [cs.CL]
  (or arXiv:2611.21650v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21650

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Himaghna Bhattacharjee [view email]
[v1] Thu, 15 Oct 2026 04:53:04 UTC (463 KB)

来源:arXiv cs.AI · arxiv.org