arXiv cs.AI· Julia Witte Zimmerman, Calla G. Beauregard, Tabia Tanzin Prama, Parisa Suchdev, Kathryn Cramer, Elisabeth Kollrack·· 6 小时前AI 评分43
研究表明 LLM 并非随机鹦鹉：人造语言任务证明其具备意义中介的抽象能力
LLMs are not stochastic parrots: Evidence for meaning-mediated abstraction from conlang-like tasks
AI 导读
一项 arXiv 研究用人造语言式任务检验“随机鹦鹉”论断：给多个 LLM 只提供虚构语言的自然语言描述，不给任何示例输出，这些语言刻意组合了训练数据中罕见或从未出现的特征。在三个任务家族中，模型系统性地朝意义预测的方向表现，能区分接触提示词与按指令使用、依新约束改变语义关系，有时还能精确匹配复杂翻译答案，据此反驳了强随机鹦鹉假说。
正文 · 原文
Abstract:The strong version of the stochastic parrot argument claims that, although large language models (LLMs) may exceed rote regurgitation, they cannot move beyond statistical pattern matching into abstraction or reasoning, remaining ontologically near the lower bound of pattern reuse despite producing alluringly fluent text. We test this hypothesis using conlang-like tasks. Several LLMs are given only natural-language descriptions of fictional languages that subvert prominent superficial patterns in training data by combining statistically uncommon and unattested features. Crucially, no example outputs are given. We argue that if the models exhibit rule-following behaviour, they cannot be relying solely on superficial statistical patterns; such patterns often work against the correct output. Instead, successful performance requires representations of the constraints specified in the prompt. Across three complementary task families, models systematically move in the meaning-predicted direction: they distinguish prompt exposure from instructed use, alter semantic relationships in response to novel constraints, and sometimes produce exact matches to complex translation answer keys. Although performance varies across the spectrum of models used, these results provide evidence for meaning-mediated abstraction in LLMs and refute the strong stochastic parrot hypothesis. Our work shows that, under appropriate architectural and contextual constraints, statistical learning can produce meaning-mediated abstractions, although generation remains strongly constrained by superficial plausibility. We discuss implications for model development and for understanding how increasingly abstract representations may emerge from plausible-text-generation objectives.
| Comments: | Presented as a poster at ISC Summer School 2026; related work presented at IC2S2 2026; a synthesis of this project and related work presented at SLSA/ 4S 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.34187 [cs.CL] |
| (or arXiv:2609.34187v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.34187 arXiv-issued DOI via DataCite |
Submission history
From: Julia Zimmerman [view email]
[v1]
Mon, 28 Sep 2026 03:03:12 UTC (11,363 KB)
[v2]
Tue, 29 Sep 2026 15:46:02 UTC (11,365 KB)
[v3]
Sun, 4 Oct 2026 22:55:09 UTC (11,367 KB)
[v4]
Sun, 25 Oct 2026 16:56:53 UTC (11,369 KB)
来源：arXiv cs.AI · arxiv.org