arXiv cs.AI· Mingda Zhang, Yuanhao Dong, Qiang Huang, Rui Qian, Yanjin Li, Zhenyu Liu, Xiao Huang, Zihao Yang, Jian Wang, Pinjia He, Xiaoying Tang·· 2 小时前AI 评分48
BaseRSI:通过免训练 Harness 内化实现基础视觉语言模型的递归自我改进
BaseRSI: Recursive Self-Improvement of a Base Vision-Language Model via Training-Free Harness Internalization
AI 导读
arXiv 论文提出 BaseRSI,一种递归自我改进新范式:基础视觉语言模型改进自身 harness,并通过免训练编辑将 harness 的作用写入自身权重。其配套机制 ReGap 用每个 transformer block 的激活差信号监督编辑,ReKey 通过回归将信号写入少量 key-value 神经元。在十个视觉推理基准上,BaseRSI 均稳定提升基础模型表现,代码已开源。
正文 · 原文
Authors:Mingda Zhang, Yuanhao Dong, Qiang Huang, Rui Qian, Yanjin Li, Zhenyu Liu, Xiao Huang, Zihao Yang, Jian Wang, Pinjia He, Xiaoying Tang
Abstract:Recursive self-improvement (RSI) is moving from the harness of prompts and tools around a base model toward the model itself. However, existing RSI still faces key challenges, including harness-bound gains that stay outside the base model, black-box supervision that reads only the model's outputs, and imprecise edits whose site and strength are fixed in advance. To address these challenges, we propose BaseRSI, a new paradigm of Recursive Harness Internalization: the base model improves its harness, then writes what the harness adds into its own weights through training-free edits, round after round on its own responses. To support this paradigm, we introduce ReGap, a mechanism that supervises every transformer block with the high-dimensional activation difference that the harness makes at that block alone, and updates the block's weights along this signal in a low-dimensional subspace. Building on this mechanism, we further propose ReKey, an algorithm that writes this signal by regression into a few key-value neurons of every block, and measures the site and strength of each edit on the model's own responses. Moreover, the base model fills every role of this recursion with forward passes alone, and its edits, written at different sites, compose in one checkpoint without refitting. Experimental results on ten visual reasoning benchmarks show that BaseRSI consistently improves a base vision-language model both in distribution and on held-out benchmarks. Our code is available at this https URL.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24578 [cs.CV] |
| (or arXiv:2611.24578v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24578 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mingda Zhang [view email]
[v1]
Wed, 18 Nov 2026 15:10:11 UTC (26,825 KB)
来源:arXiv cs.AI · arxiv.org