分数不等于结构：大脑对齐与跨语言迁移中的测量陷阱
The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer
AI 导读
arXiv 论文指出相似度分数不能证明模型与大脑或跨语言共享结构。覆盖 17 种语言的语法探测中，最远的 4 种语言准确率处于随机水平，剔除后解释方差减半；将 272 个语言对当独立样本时 p = 0.0006，以 17 种语言为单位则 p = 0.155。大脑对齐将语言模型与 fMRI 响应的相似度从 0.10 提到 0.34（可靠性上限 0.54），但破坏大脑对应关系后仍达 0.31。
正文 · 原文
Abstract:Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.
| Comments: | 13 pages, 3 figures, 2 tables. Accepted as a poster at the NeurIPS 2026 - Linguistic Principles for Foundation Models (LP4FM). Code: this https URL - revised to add more relevant references |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.03827 [cs.CL] |
| (or arXiv:2610.03827v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.03827 arXiv-issued DOI via DataCite |
Submission history
From: Saman Rahbar [view email]
[v1]
Fri, 2 Oct 2026 09:21:49 UTC (67 KB)
[v2]
Wed, 7 Oct 2026 19:00:14 UTC (67 KB)
[v3]
Fri, 16 Oct 2026 00:21:41 UTC (68 KB)
[v4]
Tue, 27 Oct 2026 02:37:13 UTC (90 KB)
来源：arXiv cs.AI · arxiv.org