arXiv cs.AI· Toni J. B. Liu, Baran Zadeo\u{g}lu, Gurbir Arora, Rapha\"el Sarfati, Nicolas Boull\'e, Christopher J. Earls·· 5 小时前AI 评分43
Jacobian Scopes：面向 LLM token 级归因的统一几何框架
Jacobian Scopes: A Unified Geometric Framework for Token-Level LLM Attribution
AI 导读
研究者提出 Jacobian Scopes，一套基于梯度与信息几何的 LLM token 级因果归因方法，可量化输入 token 对特定 logits、预测分布及模型不确定性（有效温度）的影响。
正文 · 原文
Abstract:Large language models (LLMs) make next-token predictions based on clues present in their context, such as semantic descriptions and in-context examples. Yet, elucidating which prior tokens most strongly influence a given prediction remains challenging due to the proliferation of layers and attention heads in modern architectures. We propose Jacobian Scopes, a suite of gradient-based, token level causal attribution methods for interpreting LLM predictions. Grounded in perturbation theory and information geometry, Jacobian Scopes quantify how input tokens influence various aspects of a model's prediction, such as specific logits, the full predictive distribution, and model uncertainty (effective temperature). Through case studies spanning instruction understanding, translation, and in-context learning (ICL), we demonstrate how Jacobian Scopes reveal implicit political biases, uncover word- and phrase-level translation strategies, and shed light on recently debated mechanisms underlying in-context time-series forecasting. On token-ablation faithfulness benchmarks (LAMBADA and IWSLT2017) across six LLMs, the distribution-level Fisher and Temperature Scopes significantly outperform established gradient-based attribution methods in 10 of 12 model-dataset settings. To facilitate exploration of Jacobian Scopes on custom text, we open-source our implementations and provide a cloud-hosted interactive demo at this https URL.
| Comments: | 35 pages, 17 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.16407 [cs.CL] |
| (or arXiv:2601.16407v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.16407 arXiv-issued DOI via DataCite |
|
| Journal reference: | Published as a conference paper at Neurips 2026 |
Submission history
From: Jianbang Liu [view email]
[v1]
Fri, 23 Jan 2026 02:36:38 UTC (7,914 KB)
[v2]
Sun, 15 Mar 2026 23:40:50 UTC (9,356 KB)
[v3]
Thu, 11 Jun 2026 18:04:05 UTC (9,440 KB)
[v4]
Mon, 15 Jun 2026 18:41:08 UTC (9,441 KB)
[v5]
Tue, 27 Oct 2026 21:49:55 UTC (9,462 KB)
来源：arXiv cs.AI · arxiv.org