RoSeViT：面向 ARC 视觉推理的角色分离 Vision Transformer
RoSeViT: Role-Separated Vision Transformers for ARC Visual Reasoning
AI 导读
研究者提出 RoSeViT，一种角色分离的 Vision Transformer，通过 controller tokens 推断变换规则、workspace tokens 表示视觉网格，并在结构化注意力中将非局部交互经由 controller 路由。
正文 · 原文
Abstract:ARC tasks can be formulated as visual transformation problems, where a model must infer a high-level transformation rule from a small set of demonstrations and apply it consistently across a grid. While dense Vision Transformers (ViTs) perform well in this setting, they entangle local visual state and global transformation logic within a shared token representation, which can hinder consistent reasoning. In this paper, we introduce RoSeViT, a role-separated Vision Transformer that explicitly decouples high-level instruction inference from visual workspace computation. RoSeViT uses controller tokens to aggregate transformation-level information and workspace tokens to represent the visual grid, with a structured attention pass in which nonlocal workspace interactions are routed through the controller. This design is motivated by our formal analysis showing that enforcing a shared instruction representation can reduce the effective complexity of visual transformation modeling. RoSeViT outperforms dense ViT baselines under the standard VARC training and evaluation protocol for ARC. In a controlled backbone replacement setting, it improves ARC-1 accuracy from 54.5 to 56.6, and with recurrent refinement reaches 58.8. When integrated into the full system, RoSeViT-Ensemble improves performance from 60.4 to 62.6 on ARC-1. These results demonstrate that explicitly separating instruction and workspace representations leads to more effective visual reasoning on ARC tasks. GitHub repository: this https URL.
| Comments: | NeurIPS 2026 poster (camera-ready). Code, checkpoints and predictions: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG) |
| ACM classes: | I.2.6; I.2.10 |
| Cite as: | arXiv:2601.13562 [cs.AI] |
| (or arXiv:2601.13562v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2601.13562 arXiv-issued DOI via DataCite |
Submission history
From: Zhiguang Liu [view email]
[v1]
Tue, 20 Jan 2026 03:37:17 UTC (1,813 KB)
[v2]
Wed, 28 Oct 2026 04:48:58 UTC (1,791 KB)
来源：arXiv cs.AI · arxiv.org