DiffuSpace 团队发布扩散语言模型 Clover-35B-A3B-Preview 技术报告
Clover-35B-A3B-Preview Technical Report
AI 导读
DiffuSpace 技术团队发布 Clover-35B-A3B-Preview 技术报告,该模型为 350 亿总参数、30 亿激活参数的 MoE 扩散语言模型,支持思考模式。
推荐理由
原文给出扩散语言模型的架构细节与多项智能体基准数字,读者可据此评估其在受限显存部署下的实际表现。
正文 · 原文
Abstract:We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.
| Comments: | Technical Report. Code: this https URL Weights: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.24636 [cs.CL] |
| (or arXiv:2611.24636v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24636 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shansan Gong [view email]
[v1]
Wed, 18 Nov 2026 15:46:14 UTC (423 KB)
来源:arXiv cs.AI · arxiv.org