评估与改进测序协议理解 AI 智能体：LibStructBench 基准
Evaluating and improving AI agents for sequencing protocol understanding
AI 导读
研究者发布 LibStructBench，一个基于文档的基准，用于评估 AI 智能体能否按测序协议精确重构测序文库结构与分子工作流。实验显示前沿智能体常产出近似正确的重构，但在分子态表示和工作流连接性等精确重构上仍不可靠；更换底层语言模型带来的平均性能差异大于更换智能体框架，仅分子态重构差异具统计显著性。结果显示需要更可控、可验证的分子过程科学推理方法。
正文 · 原文
Abstract:Existing scientific-agent benchmarks rarely evaluate whether agents can reconstruct the exact molecular structures and generation workflows specified by sequencing protocols, a capability critical for analyzing customized sequencing assays. We introduce LibStructBench, a document-grounded benchmark for reconstructing sequencing-library structures and molecular workflows. Our experiments show that frontier agents often produce approximately correct reconstructions but remain unreliable at exact reconstruction, particularly in molecular-state representation and workflow connectivity. Switching the underlying language model produced larger average performance differences than switching agent harnesses, with a statistically significant difference only for molecular-state reconstruction. Additional knowledge improved oligo-family recovery in a small pilot without improving mean molecular-workflow scores. Autonomous accumulation of procedural rules and cross-protocol examples showed non-monotonic validation performance and no mean held-out molecular-workflow improvement in the tested trajectory. These findings highlight the need for more controlled and verifiable approaches to scientific reasoning about molecular processes in complex experimental protocols.
| Subjects: | Artificial Intelligence (cs.AI); Genomics (q-bio.GN) |
| Cite as: | arXiv:2610.37851 [cs.AI] |
| (or arXiv:2610.37851v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37851 arXiv-issued DOI via DataCite |
Submission history
From: Chi-Lam Poon [view email]
[v1]
Tue, 27 Oct 2026 18:58:58 UTC (554 KB)
来源：arXiv cs.AI · arxiv.org