蛋白质语言模型在氨基酸字母重标记下无法保持接触预测
Protein Language Models Fail To Capture Contact Predictions Under Invariant Relabeling
AI 导读
一项入选 NeurIPS 2026 ICBINB 工作坊的研究提出对氨基酸字母表做全局双射重标记的诊断框架，检验蛋白质语言模型（pLM）是否真正学到不变的共进化统计。结果显示 Potts 模型等经典方法在该变换下保持不变，而 ESM-2、MSA Transformer、AMPLIFY 等 8M 至 3B 参数的 pLM 接触图谱均崩溃，表明其依赖显式的残基身份（"字母表锚定"）而非纯共进化信号。
正文 · 原文
Abstract:Protein language models (pLMs) produce highly accurate contact maps from single-sequence inputs. People often take this accuracy as proof that these models implicitly learn invariant coevolutionary statistics, extracting the same signals as classical alignment-based methods like Potts. In this work, we introduce a diagnostic framework around a global bijective relabeling of the amino-acid alphabet. This transformation permutes residue letters while leaving all alignment-level information, column entropies, and pairwise edit distances unchanged. Pseudolikelihood Potts models and inverse-covariance estimators remain invariant under this transformation, but pLMs fail the test. Across multiple architectures (ESM-2, MSA Transformer, AMPLIFY) and model scales (8M to 3B parameters), pLM contact maps collapse under arbitrary relabeling. Using Hamming distance controls and biochemical grouping interventions, we attempt to isolate what likely causes this degradation and find that these models rely on explicit, canonical residue identities (``alphabet anchoring'') rather than pure coevolutionary statistics. Increasing scale makes models more responsive to biochemical classes, but their performance still degrades under relabeling. Our findings show a gap between how pLMs and classical coevolutionary models compute contacts. More broadly, our work shows how diagnostic perturbations that leave classical models provably unchanged can expose hidden representation biases and prevent over-reliance on neural black boxes in biological modeling.
| Comments: | Accepted to the ICBINB - Failure Modes of AI in Biology Workshop at NeurIPS 2026. 15 pages, 6 figures |
| Subjects: | Quantitative Methods (q-bio.QM); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.16881 [q-bio.QM] |
| (or arXiv:2611.16881v1 [q-bio.QM] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16881 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aryan Keluskar [view email]
[v1]
Thu, 12 Nov 2026 14:45:07 UTC (3,318 KB)
来源：arXiv cs.AI · arxiv.org