用稀疏自编码器解析 ESMFold 折叠主干的结构机制
Structure, Not Sequence: Tracking and Validating Folding Mechanisms in ESMFold's Trunk with Sparse Autoencoders
AI 导读
研究者用稀疏自编码器（SAEs）对蛋白质结构预测模型 ESMFold 的折叠主干进行三步可解释性分析：定位主干中的空间计算分布，揭示 single representation 携带继承语义、pair representation 计算成对几何的分工，并用推理语言模型大规模解读字典、以梯度归因追踪概念跨模块组合。
正文 · 原文
Abstract:Structure predictors like ESMFold fold proteins using neural network computation and have transformed structural biology. However, the driving mechanism from amino-acid (AA) sequence to 3D structure is insufficiently understood, and biological meaning remains locked in the dense network. Sparse autoencoders (SAEs) have shown promising results in obtaining interpretable concept features over an overcomplete basis of intermediate protein sequence-model representations. Yet folding models, which predict physical shape from AA sequence semantics, have no such account. We hypothesize that the features carrying the folding computation evolve in the 3D structural regression process. In this paper, we interpret the folding trunk in three steps. First, a pilot analysis on the representation distribution across modules and recycles locates distinct spatial computation in the trunk. Second, the dictionaries expose a division of labor: the single representation carries inherited semantics, and the pair representation computes the pairwise geometry. Third, we interpret the dictionary at scale with a reasoning language model, trace how the concepts compose across blocks by gradient-based attribution, and follow the interpreted code to the model's folding output, verifying where the computation aligns with, or departs from, known biochemistry. Together, the sparse dictionaries we train capture the relational geometry the trunk computes beyond the sequence model, and their concept activations forecast the model's own fold quality, turning the black-box folding trunk into a sparse, interpretable, and scientifically verifiable circuit.
| Subjects: | Quantitative Methods (q-bio.QM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.36226 [q-bio.QM] |
| (or arXiv:2610.36226v1 [q-bio.QM] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36226 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bohan Liu [view email]
[v1]
Mon, 26 Oct 2026 20:04:57 UTC (1,204 KB)
来源：arXiv cs.AI · arxiv.org