arXiv cs.AI· Chenlu Ding, Jiancan Wu, Leheng Sheng, Fan Zhang, Yancheng Yuan, Xiang Wang, Xiangnan He·· 4 小时前AI 评分30
MLLMEraser：通过激活导向在多模态大语言模型中实现测试时遗忘
MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering
AI 导读
研究者提出 MLLMEraser，一个无需训练的输入感知测试时遗忘框架，通过激活导向在多模态大语言模型中实现知识擦除，无需参数更新。该方法利用对抗性扰动构建多模态擦除方向，并由输入感知机制自适应决定何时施加擦除，以保留剩余知识效用。在 LLaVA-1.5 和 Qwen-2.5-VL 上的实验表明，MLLMEraser 在遗忘效果上超越现有基线，计算成本更低且效用损失最小。
正文 · 原文
Abstract:Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and harmful content. Existing unlearning approaches for MLLMs typically adapt training-based strategies such as gradient ascent or preference optimization, but these methods are computationally expensive, irreversible, and often distort retained knowledge. In this work, we propose MLLMEraser, an input-aware, training-free framework for test-time unlearning. Our approach leverages activation steering to enable dynamic knowledge erasure without parameter updates. Specifically, we construct a multimodal erasure direction by contrasting adversarially perturbed, knowledge-recall image-text pairs with knowledge-erasure counterparts, capturing both textual and visual discrepancies. To prevent unnecessary interference, we further design an input-aware steering mechanism that adaptively determines when and how the erasure direction should be applied, preserving utility on retained knowledge while enforcing forgetting on designated content. Experiments on LLaVA-1.5 and Qwen-2.5-VL demonstrate that MLLMEraser consistently outperforms state-of-the-art MLLM unlearning baselines, achieving stronger forgetting performance with lower computational cost and minimal utility degradation.
| Comments: | Accepted at NeurIPS 2026 (40th Conference on Neural Information Processing Systems) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2510.04217 [cs.LG] |
| (or arXiv:2510.04217v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2510.04217 arXiv-issued DOI via DataCite |
Submission history
From: Chenlu Ding [view email]
[v1]
Sun, 5 Oct 2025 14:20:17 UTC (891 KB)
[v2]
Fri, 10 Oct 2025 11:47:30 UTC (890 KB)
[v3]
Mon, 2 Feb 2026 12:41:26 UTC (965 KB)
[v4]
Fri, 30 Oct 2026 08:25:33 UTC (1,090 KB)
来源：arXiv cs.AI · arxiv.org