arXiv cs.AI· Soorya Ram Shimgekar, Agam Goyal, Amruta Parulekar, Joshua Chen, Yian Wang, Navin Kumar, Hari Sundaram, Eshwar Chandrasekharan, Koustuv Saha·· 4 小时前AI 评分43
Prompt PerturbAItions：大语言模型回答不一致的机制研究
Prompt PerturbAItions: Mechanisms of LLM Response Inconsistencies
AI 导读
一项 arXiv 预印本研究了词汇和语气层面的提示词扰动如何影响 LLM 的事实可靠性，在礼貌、随机和逐渐有毒的受控变体下评测七个 LLM 的 ARC-Easy、GSM8K 和 MMLU 表现。结果显示大多数模型的准确率下降、预测不确定性上升，且毒性越高准确率越低。归因图分析发现扰动会放大对扰动敏感的 Variant 节点相对于核心推理节点的激活与影响，且随毒性分数单调上升。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly deployed in conversational settings where user phrasing ranges from polite to casual to toxic, yet less is known about whether such lexical variation in otherwise semantically equivalent prompts can degrade factual reliability. We study how lexical and tone-based prompt perturbations affect the factual reliability of LLMs. Using controlled prompt variations spanning polite, random, and increasingly toxic levels, we evaluate seven LLMs on ARC-Easy, GSM8K, and MMLU. We find that lexical perturbations reduce factual accuracy and increase predictive uncertainty in most models. Interestingly, increasing toxicity results in decreasing accuracy. To examine whether these answer inconsistencies correspond to internal changes, we conduct attribution-graph analyses of model activations and influences. We find that perturbations amplify perturbation-sensitive variant nodes relative to core reasoning nodes. We also observed a monotonic trend with increasing toxicity score and increasing activation and influence in Variant nodes. These findings position prompt wording and tone as a critical dimension of LLM reliability and provide behavioral and mechanistic evidence that surface-level lexical variation can alter factual outputs and internal computation.
| Comments: | Preprint: 16 pages, 17 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2605.30913 [cs.CL] |
| (or arXiv:2605.30913v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.30913 arXiv-issued DOI via DataCite |
Submission history
From: Agam Goyal [view email]
[v1]
Fri, 29 May 2026 06:58:47 UTC (51 KB)
[v2]
Mon, 26 Oct 2026 16:02:17 UTC (246 KB)
来源：arXiv cs.AI · arxiv.org