基准测试预训练分子嵌入模型的分子表示学习能力
Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning
AI 导读
一项新研究在统一框架下用分层贝叶斯 Bradley-Terry 统计检验评估了 28 个分子嵌入模型在 25 个分类数据集上的表现。结果发现除 CheMeleon 外所有神经模型都未能超越经典 ECFP count 指纹，而基于 scaffold 交叉验证的外部检验也未显示哪类嵌入有确定优势。
正文 · 原文
Abstract:Pretrained molecular embedding models spanning graph neural networks, text transformers, and hybrid architectures have been widely used in drug discovery, yet no large-scale, rigorous comparison of their representations existed. We evaluated 28 models across 25 classification datasets under a controlled framework using hierarchical Bayesian Bradley-Terry statistical testing. All neural models except CheMeleon failed to outperform the classical ECFP count fingerprint; CheMeleon was the only model rated as better. However, further testing on external validation datasets, based on scaffold cross-validation and a Bayesian hierarchical correlated t-test, revealed no definite advantage of either embedding method. As a practical takeaway, we recommend evaluating both ECFP fingerprints and CheMeleon embeddings in novel QSAR/QSPR studies. Overall, in the context of all neural networks evaluated, these results also challenge the perceived progress in molecular representation learning and highlight the need for rigorous evaluation standards in the field.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2508.06199 [cs.LG] |
| (or arXiv:2508.06199v5 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2508.06199 arXiv-issued DOI via DataCite |
Submission history
From: Mateusz Praski [view email]
[v1]
Fri, 8 Aug 2025 10:29:24 UTC (2,277 KB)
[v2]
Wed, 13 Aug 2025 06:01:26 UTC (2,277 KB)
[v3]
Sun, 14 Sep 2025 11:09:51 UTC (2,277 KB)
[v4]
Mon, 23 Feb 2026 14:41:11 UTC (2,286 KB)
[v5]
Tue, 3 Nov 2026 20:20:40 UTC (3,820 KB)
来源：arXiv cs.AI · arxiv.org