arXiv cs.AI· Yile Gu, Zhen Zhang, Shaowei Zhu, Xinwei Fu, Jun Wu, Yida Wang, Baris Kasikci·· 4 小时前AI 评分50
Ekka：LLM 推理中静默错误的自动化诊断系统
Ekka: Automated Diagnosis of Silent Errors in LLM Inference
AI 导读
Ekka 是一个自动化诊断系统，通过对比目标与参考框架的中间执行状态，定位 LLM serving 框架中无报错却导致输出质量下降的静默错误根因。在基于主流 serving 框架真实错误构建的 benchmark 上，Ekka 达到 80% pass@1 和 88% pass@5 诊断准确率，超过 SOTA 系统，并新诊断出 4 个已被开发者确认的静默错误。
正文 · 原文
Abstract:LLM serving frameworks are quickly evolving with a complex software stack and a vast number of optimizations. The rapid development process can introduce silent errors where output quality silently degrades without any explicit error signals. Diagnosing silent errors is notoriously difficult due to the substantial semantic gap between the high-level symptoms and the low-level root causes. We observe that diagnosis of silent errors can be effectively framed as a differential debugging problem by leveraging the existence of semantically correct reference implementations. We propose Ekka, an automated diagnosis system that identifies root causes by systematically aligning and comparing intermediate execution states between a target and a reference framework. We constructed a benchmark of real-world silent errors from popular serving frameworks, where Ekka shows 80% pass@1 diagnosis accuracy and 88% pass@5 diagnosis accuracy, outperforming state-of-the-art systems. Ekka also diagnoses 4 new silent errors from serving frameworks, all of which have been confirmed by the developers.
| Comments: | ICML 2026, open-source at this https URL |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Software Engineering (cs.SE) |
| ACM classes: | D.4.5; I.2.1 |
| Cite as: | arXiv:2606.04594 [cs.DC] |
| (or arXiv:2606.04594v2 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2606.04594 arXiv-issued DOI via DataCite |
Submission history
From: Yile Gu [view email]
[v1]
Wed, 3 Jun 2026 08:32:13 UTC (1,140 KB)
[v2]
Tue, 3 Nov 2026 23:54:41 UTC (1,140 KB)
来源：arXiv cs.AI · arxiv.org