跳到正文
arXiv cs.AI· Addi Malviya-Thakur, Reed Milewicz, Gregory R. Watson, Audris Mockus·· 1 小时前AI 评分25

AI 时代的科研软件评估:研究软件指标全景综述

Measuring What Matters in the Age of AI: The Research Software Metrics Landscape

AI 导读

arXiv 论文对研究软件指标做全景综述,基于 49 篇参考文献将常见指标归为八大类,按数据来源、构念、效度证据、自动化潜力和滥用风险进行描述。作者指出 AI 辅助开发使代码产出速度超过人工审查能力,溯源归因和学术署名体系难以跟上,多数被广泛引用的指标只是局部信号,单独衡量科研影响力价值有限。论文呼吁负责任、情境化、混合方法地使用指标,并为 RSE、PI、机构和资助者提供选型建议。

正文 · 原文

View PDF HTML (experimental)

Abstract:Research software now serves as a primary instrument of science, but the methods the community uses to judge its quality, sustainability, reuse, and impact remain scattered across software engineering, open science, FAIR software, citation, and research-assessment literature. AI-assisted development sharpens the problem: code can be produced at rates that outpace human review capacity, provenance and attribution are harder to track, scholarly credit systems cannot keep pace, and existing metrics inherit assumptions about how research software gets developed that may no longer apply. To aid in navigating these challenges, we present a focused landscape synthesis of research software metrics based on 49 substantive references, organizing the recurring metrics into eight families described by data source, claimed construct, evidence of validity, automation potential, and susceptibility to misuse. Many widely cited research software metrics serve as partial signals but make weak standalone measures of scientific impact. The AI era opens or widens gaps around provenance, maintainability of generated code, reviewer effort, model and data dependencies, reuse and community-health signal inflation, and credit for human labor that AI does not perform. We argue for responsible, contextual, mixed-method use of research software metrics, and offer decision guidance for Research Software Engineers (RSEs), principal investigators, institutions, and funders selecting metrics that support better software, fairer credit, and more trustworthy science. All findings rest on the 49-reference focused review; a reproducible search-and-triage protocol and a 727-record triaged corpus accompany the paper as a transparency artifact for a future full systematic mapping study.
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG)
Cite as: arXiv:2611.23253 [cs.SE]
  (or arXiv:2611.23253v1 [cs.SE] for this version)
  https://doi.org/10.48550/arXiv.2611.23253

arXiv-issued DOI via DataCite

Submission history

From: Addi Malviya Thakur [view email]
[v1] Fri, 25 Sep 2026 18:48:56 UTC (50 KB)

来源:arXiv cs.AI · arxiv.org