XAI 研究揭示：解释正确性并非都影响人类理解
Does Explanation Correctness Matter? Linking Computational XAI Evaluation to Human Understanding
AI 导读
一项用户研究（N=200）考察可解释 AI（XAI）的功能正确性指标是否对应人类理解。在合成时间序列分类任务中，解释正确性设为 100%、85%、70%、55% 四档，以 forward simulation 准确率衡量理解程度：70% 和 55% 时准确率相对 100% 下降，85% 与 100% 无显著差异。
正文 · 原文
Abstract:Explainable AI (XAI) methods are commonly evaluated using functional correctness metrics, also called faithfulness or fidelity, which estimate how closely an explanation reflects the model's reasoning. Higher correctness is assumed to produce better human understanding, but this link has not been tested with controlled levels. We conducted a user study (N=200) that manipulated explanation correctness at four levels (100%, 85%, 70%, 55%) in a synthetic time series classification task where participants could not rely on domain knowledge or visual intuition. Explanations were constructed to resemble feature attributions, with correctness defined against a known ground truth rather than estimated from a trained model. Participants predicted a simulated AI's decisions from these explanations (forward simulation), and we used their forward simulation accuracy as a proxy for understanding. Correctness affected forward simulation accuracy, but not at every level: accuracy dropped at 70% and 55% relative to 100%, with no further reduction below 70% and no conclusive difference between 85% and 100%. Lower correctness reduced how many participants predicted accurately rather than lowering accuracy uniformly, and even with fully correct explanations, only a subset of participants predicted well above chance. In exploratory analyses, some participants predicted decisions accurately but described the wrong pattern when asked what the AI used, and self-reported ratings correlated with forward simulation accuracy only when explanations were fully correct. These findings suggest that, for feature-attribution-style explanations, not all differences in functional correctness translate to differences in human understanding, highlighting the need to validate functional metrics against human outcomes.
| Comments: | 33 pages, 11 figures, 3 tables |
| Subjects: | Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2603.25251 [cs.HC] |
| (or arXiv:2603.25251v3 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2603.25251 arXiv-issued DOI via DataCite |
Submission history
From: Gregor Baer [view email]
[v1]
Thu, 26 Mar 2026 09:55:10 UTC (401 KB)
[v2]
Tue, 11 Aug 2026 11:29:30 UTC (451 KB)
[v3]
Wed, 4 Nov 2026 09:05:09 UTC (452 KB)
来源：arXiv cs.AI · arxiv.org