归因 Agent 失败需要执行反事实：迈向一种报告标准

View PDF HTML (experimental)

Abstract:Published explanations of why long-horizon agents fail are commonly taxonomies labeled over completed trajectories: a judge reads a transcript and names the failure. We argue that a claim attributing an agent's behavior to its history should carry intervention-verified evidence, per instance or via a validated estimator: freeze an executable mid-trajectory state, alter only the history the agent sees, and continue - executed rather than inferred from the transcript, at a state whose restoration has been measured, with a replay certificate matched to the estimand. Each of the four manipulation classes we name has a precedent, and several methods already execute interventions from checkpoints, re-executed prefixes or recorded conversation states (for example DoVer, REFLECT, Min et al., Who&When Pro, Causal Agent Replay), but none reports a measured replay certificate. Two cautionary measurements, neither applying the standard, follow: a carrier taxonomy whose "concentration" could not be separated from benchmark construction (adjusted odds ratio 1.15, interval crossing 1), and hash-identical snapshots that, restored into identically seeded processes, agreed with one another in 24 of 24 states while differing from the live continuation in 9 and 20 of 24. Restoring the captured position of a random stream made all 24 agree with the live run.

Comments:	Accepted at the IAB Workshop (Interpreting Agent Behavior) at NeurIPS 2026 (non-archival). 20 pages
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2611.05650 [cs.AI]
	(or arXiv:2611.05650v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2611.05650 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Egor Pakhomov [view email]
[v1] Tue, 3 Nov 2026 21:33:48 UTC (52 KB)