CUSD 模型合并方法：分离共同与独有方向提升合并效果
Separating Common and Unique Directions for Model Merging
AI 导读
一项发表于 NeurIPS 2026 的研究提出 Common–Unique Subspace Decomposition（CUSD），一种免数据的模型合并预处理方法，将任务向量拆分为共同方向分量与独有残差分别处理。实验覆盖 11 个基础模型、四个模型家族和四种算术合并算子，CUSD 改善了 44 个模型-算子组合中的 42 个，相比对应基线最高提升 22.5%。代码已公开。
正文 · 原文
Abstract:Model merging combines multiple fine-tuned models without additional training, and arithmetic methods have emerged as a simple yet effective way to merge models without data. However, existing arithmetic methods treat task vectors in parameter space and apply a single element-wise rule to directions that can differ in mergeability. In a shared directional basis, we find a consistent two-regime structure: a low-dispersion common bulk shared across tasks and a high-dispersion task-dominated tail that carries disproportionate energy. These findings motivate Common--Unique Subspace Decomposition (CUSD), a data-free pre-merging method that separates task vectors into common and unique directional components, merges the common components with a downstream arithmetic operator, and retains unique residuals separately. Experiments across $11$ base models, four model families, and four arithmetic-based merging operators show that CUSD improves 42 of 44 model--operator combinations and achieves up to 22.5% improvement over the corresponding baselines. Our code is publicly available at this https URL
| Comments: | Accepted by NeurIPS 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.04108 [cs.LG] |
| (or arXiv:2611.04108v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.04108 arXiv-issued DOI via DataCite |
Submission history
From: Shiting Wang [view email]
[v1]
Tue, 3 Nov 2026 00:32:11 UTC (1,000 KB)
[v2]
Wed, 4 Nov 2026 10:47:34 UTC (1,000 KB)
来源：arXiv cs.AI · arxiv.org