HydroMind:面向自主水文研究的可检验 Agent
HydroMind: A Checkable Agent for Autonomous Hydrological Research
AI 导读
研究者开发了水文建模系统 HydroMind,并比较三种 agent harness(工具与指令框架)对同一大模型智能体自主水文研究的影响。三种 harness 下智能体分别提出 23、20、41 条断言,其中不成立的比例为 30%、20%、5%,可检验(报告统计检验、置信区间或判定规则)的比例为 48%、53%、83%。
正文 · 原文
Abstract:General-purpose coding agents have written code for hydrological and Earth-system models. In these studies the researchers decided in advance how the model would be evaluated, for example the test period, the scoring function, and the standard the model had to meet. No study has tested what happens when the agent makes these decisions itself: how many of its claims are supported by its own data, code, and results, and whether readers can check them. Both may depend on the agent's harness, that is, the tools it can call and the written instructions it follows. We compared three harnesses: the agent's own harness; this harness plus the modeling tools of HydroMind, a hydrological modeling system we developed for general-purpose agents, which run every model through fixed commands and store its settings, trained model, and predictions; and both plus HydroMind's research procedure, under which the agent writes a research design before running any model and records every change to it. Under each harness, the same agent studied four open questions in hydrological model development and was given only the data. Under the three harnesses, the experiments submitted 23, 20, and 41 claims, each stating that one model or approach is better than, worse than, or similar to another. A claim does not hold if the difference recomputed from the experiment's own code and predictions disagrees with it, or if the code did not compare the two models or approaches the claim names. Under the three harnesses, 30%, 20%, and 5% of the claims did not hold. A statistical test, confidence interval, or decision rule that lets readers judge the stated difference was reported for 48%, 53%, and 83% of the claims. With the language model unchanged, changing only the harness reduced the share of claims that do not hold and increased the share that readers can check.
| Comments: | 16 pages, 4 figures, 1 table |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Geophysics (physics.geo-ph) |
| Cite as: | arXiv:2611.23323 [cs.SE] |
| (or arXiv:2611.23323v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23323 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zheng Fang [view email]
[v1]
Sun, 15 Nov 2026 17:53:30 UTC (778 KB)
来源:arXiv cs.AI · arxiv.org