AI 系统的可靠性工程：挑战、方法与方向
Reliability Engineering for AI Systems: Challenges, Methods, and Directions
AI 导读
Rong Pan 在 arXiv 论文中将 FMEA、加速测试、现场监测、可靠性增长等成熟可靠性工程方法适配到 AI 系统。论文提出四级诊断框架，把故障分为组件、运营闭环、智能体行为及网络与治理故障，并以 TEVV、序贯监测、FRACAS 与 SMART 框架创建和更新证据。结论指出既有原则仍可用作基础，但随着系统快速自我演化，亟需新的度量手段与安全护栏。
正文 · 原文
Abstract:AI reliability concerns whether an AI system performs its intended function dependably over a stated period and under stated operating conditions, with clear evidence. As these systems become more autonomous, this function should include more than only a correct output. Retrieval, memory, tool use, permissions, human oversight, and interactions among systems must operate consistently and safely, and, for a generative AI, so must the reasoning process that produces the output. Average benchmark accuracy only measures system capability; it does not quantify this broader reliability claim. This paper adapts established reliability engineering methods, from failure definitions and operational envelopes to FMEA, accelerated testing, field monitoring, and reliability growth, to AI systems. A four-level diagnostic framework classifies failures as component, operational-loop, agentic-conduct, or network and governance failures. Test, evaluation, verification, and validation (TEVV), sequential monitoring, and FRACAS create and refresh evidence. The SMART framework provides statistical guidance for measurement, analysis, assessment, and test planning, and the NIST's AI risk management framework provides organizational guidance for governance, evaluation, monitoring, and mitigation. Three cases are discussed to illustrate these programs. It is shown that while the established reliability engineering principles can still provide a usable foundation, new measurements and safety guardrails are urgently needed as these systems are self-evolving at a fast pace.
| Subjects: | Artificial Intelligence (cs.AI); Applications (stat.AP) |
| MSC classes: | 86T05, 86T01 |
| ACM classes: | A.1; I.2 |
| Cite as: | arXiv:2609.35316 [cs.AI] |
| (or arXiv:2609.35316v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.35316 arXiv-issued DOI via DataCite |
Submission history
From: Rong Pan [view email]
[v1]
Mon, 28 Sep 2026 14:48:59 UTC (56 KB)
[v2]
Sun, 1 Nov 2026 03:01:54 UTC (812 KB)
[v3]
Tue, 3 Nov 2026 22:27:19 UTC (812 KB)
来源：arXiv cs.AI · arxiv.org