面向视觉-语言-动作策略的事件锚定稀疏自编码器可解释性方法
Event-Grounded Sparse Autoencoders for Vision-Language-Action Policies
AI 导读
一项 NeurIPS 2026 研究提出事件锚定的 VLA 策略可解释性流程，将 SAE 特征分析锚定到机器人闭环行为事件而非文本上下文，通过末端执行器轨迹关键帧提取与事件聚类来排序 SAE 特征。在 OpenVLA 两个仿真架构和真机实验中，该排序产生最强因果效应，并可迁移到 π₀.₅ 的连续动作块。研究同时指出 SAE 是不完美的干预基底，其可用性随架构与干预位置变化。
正文 · 原文
Abstract:Vision-Language-Action (VLA) policies translate language and visual inputs into robot actions, where their hidden representations directly shape closed-loop behavior. However, mechanistic interpretability tools from language and vision-language models do not transfer cleanly to VLAs: outputs are robot actions rather than human-readable tokens, and validating internal changes requires expensive closed-loop rollouts. We propose an event-grounded interpretability pipeline that anchors SAE feature analysis to behavioral events rather than text contexts. We extract keyframes from end-effector trajectories and cluster them within each task using visual, robot-state, and temporal cues. The resulting events guide SAE feature ranking, with optional VLM annotations providing descriptions of the events. To our knowledge, our pipeline is among the first to ground SAE-based VLA analysis in closed-loop behavioral events. Across two simulation architectures and a real-robot study, event-grounded ranking yields the strongest causal effects on OpenVLA and transfers to the continuous action chunks of $\pi_{0.5}$. SAE is a sparse but imperfect intervention basis: usability varies with architecture and intervention site, and aggressive intervention reveals safety and interpretability limits. Overall, event-grounded SAE analysis emerges as a practical starting point for behavior-anchored VLA interpretability, motivating future work on SAE analysis beyond end-effector motion, finer-grained closed-loop evaluation, and safe interventions for high-stakes VLA deployments.
| Comments: | NeurIPS 2026 camera-ready version |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.17204 [cs.RO] |
| (or arXiv:2605.17204v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2605.17204 arXiv-issued DOI via DataCite |
Submission history
From: Xinchen Jin [view email]
[v1]
Sun, 17 May 2026 00:20:17 UTC (18,159 KB)
[v2]
Thu, 5 Nov 2026 04:44:57 UTC (16,423 KB)
来源：arXiv cs.AI · arxiv.org