arXiv cs.AI· Changxu Ni, Feiran You, Yongan Guo, Hongyang Du, Kang Wei, Le Liang, Yiyang Ni, Jun Li·· 3 小时前AI 评分35
CORMA：基于约束路由的 Decision Mamba，实现可泛化的多约束离线安全强化学习
CORMA: Constraint-Routed Decision Mamba for Generalizable Multi-Constraint Offline Safe Reinforcement Learning
AI 导读
论文提出 CORMA，一种用于多约束离线安全强化学习的约束路由 Decision Mamba 框架。其将策略推理分解为约束感知的 RTG 估计与自适应动作生成，采用专家组合融合机制和三阶段优化策略。在 OSRL 基准上，CORMA 取得更优的回报-安全权衡，推理延迟较 DT 方法降低 37%，并对未见过的安全预算组合展现出更强的零样本泛化能力。
正文 · 原文
Abstract:Offline safe reinforcement learning (RL) learns safety-aware policies from static datasets while satisfying safety constraints during deployment. Motivated by the strong trajectorymodeling capabilities of Transformers, recent Decision Transformer (DT)-based methods formulate offline safe RL as conditional sequence modeling by conditioning action generation on return-to-go (RTG) and cost-to-go (CTG) tokens, thereby enabling adaptation to varying safety budgets. However, existing DT-based methods are developed for single-constraint tasks. For tasks with multiple constraints, generalizing learned policies to unseen combinations of safety budgets remains challenging due to the conflicting requirements imposed by heterogeneous constraints. Moreover, the quadratic complexity of Transformer architectures significantly increases the computational overhead in multi-constraint tasks, as representing additional constraints requires more CTG tokens. Recently, Mamba has emerged as an efficient sequence-modeling backbone, offering linear computational complexity and strong generalization capabilities. In this paper, we propose CORMA, a constraint-routed decision Mamba framework for generalizable multi-constraint offline safe RL. CORMA decomposes policy inference into constraint-aware RTG estimation and adaptive action generation, where constraint-specific experts model feasible RTG distributions and a router-guided product-of-experts mechanism fuses these distributions to provide safety-aware guidance for a Mamba-based action decoder. A three-stage optimization strategy is designed to improve training stability. Extensive experiments on OSRL benchmarks demonstrate that CORMA achieves superior reward-safety trade-offs, reduces inference latency by 37% compared with DT-based methods, and exhibits stronger zero-shot generalization to unseen safety-budget combinations than offline safe RL baselines.
| Comments: | 30pages, 6 figures, conference |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.34208 [cs.LG] |
| (or arXiv:2610.34208v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34208 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Changxu Ni [view email]
[v1]
Sun, 25 Oct 2026 10:54:34 UTC (1,261 KB)
来源：arXiv cs.AI · arxiv.org