跳到正文
arXiv cs.AI· Dohyun Kim, Jaeung Lee, Donghee Hong, Jaemin Jo·· 4 小时前AI 评分34

DECODE:诊断自动化解释流程中的可解释性失效

DECODE: Diagnosing Interpretability Failures in Automated Interpretation Process

AI 导读

研究者提出可视化分析系统 DECODE,用于诊断 LLM 自动化解释流程的可靠性问题。该系统将诊断流程分解为三个渐进阶段,利用独立生成解释之间的一致性作为可靠性代理,并通过三阶段主动播种协议推荐高优先级实例供人工审查。在 16,384 个 Sparse Autoencoder 特征上的案例研究显示,该方法能大规模高效识别并归因可解释性失效。

正文 · 原文

View PDF HTML (experimental)

Abstract:Tools for interpreting the internal representations of Large Language Models (LLMs) increasingly rely on LLMs themselves to both generate and evaluate explanations, creating a self-referential automated interpretation process with minimal human oversight. This automation raises significant reliability concerns, which are exacerbated by the absence of ground truth. In practice, it is difficult to validate explanations or to attribute interpretability failures to their source -- whether they originate from the tool's extracted representations or from the LLM used in the process. To address these challenges, we present DECODE, a visual analytics system that reintroduces human oversight into the automated interpretation process. Our approach decomposes the diagnostic workflow into three progressive stages, isolating failure sources ranging from structural defects to explanation-level misalignments. Drawing inspiration from inter-coder consensus in qualitative analysis, we utilize consensus among independently generated explanations and their evaluation metrics as a proxy for reliability. To further scale human inspection, we employ a three-phase active seeding protocol that strategically recommends high-priority instances for manual review. Finally, we demonstrate the effectiveness of our approach through a case study on 16,384 Sparse Autoencoder features, showing that it can efficiently identify and attribute interpretability failures at scale.
Subjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
Cite as: arXiv:2611.17856 [cs.AI]
  (or arXiv:2611.17856v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17856

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jaemin Jo [view email]
[v1] Fri, 13 Nov 2026 07:40:51 UTC (2,966 KB)

来源:arXiv cs.AI · arxiv.org