arXiv cs.AI· Nathaniel S. Keplinger, Baiting Luo, Yunuo Zhang, Kyle Hollins Wray, Aron Laszka, Abhishek Dubey, Ayan Mukhopadhyay·· 2 小时前AI 评分34
NS-Gym:面向非平稳马尔可夫决策过程的开源综合仿真框架
NS-Gym: A Comprehensive and Open-Source Simulation Framework for Non-Stationary Markov Decision Processes
AI 导读
NS-Gym 是首个专为非平稳马尔可夫决策过程(NS-MDP)设计的仿真工具包,集成于 Gymnasium 框架内并开源。它将环境参数演化与智能体决策模块分离,提供标准化接口和基准问题,实现非平稳条件下算法的可复现评估。该成果发表于 NeurIPS 2025 Datasets and Benchmarks Track,作者还用它对多种已有算法进行了基准测试。
正文 · 原文
Abstract:Many real-world applications require decision-making where the environmental dynamics evolve over time. These non-stationary environments pose significant challenges to traditional decision-making models, which typically assume stationary dynamics. Non-stationary Markov decision processes (NS-MDPs) offer a framework to model and solve decision problems under such changing conditions. However, there are no standardized simulation frameworks for NS-MDPs, as opposed to widely popular frameworks for stationary problems. We present NS-Gym, the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework. In NS-Gym, we segregate the evolution of the environmental parameters that characterize non-stationarity from the agent's decision-making module, allowing for modular and flexible adaptations to dynamic environments. We review prior work in this domain and present a toolkit encapsulating key problem characteristics and types in NS-MDPs. This toolkit is the first effort to develop a set of standardized interfaces and benchmark problems to enable consistent and reproducible evaluation of algorithms under non-stationary conditions. We also benchmark several algorithmic approaches from prior work on NS-MDPs using NS-Gym. We envision that NS-Gym will enable researchers to study decision-making under non-stationarity by providing standardized interfaces, simulation frameworks, and benchmark problems. Project documentation, webpage, and code can be found at this https URL and this https URL
| Comments: | NeurIPS 2025 Datasets and Benchmarks Track. 34 pages, 26 figures, 20 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2501.09646 [cs.AI] |
| (or arXiv:2501.09646v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2501.09646 arXiv-issued DOI via DataCite |
|
| Journal reference: | Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track, pp. 1804-1844 |
| Related DOI: | https://doi.org/10.52202/085713-0059
DOI(s) linking to related resources |
Submission history
From: Nathaniel Keplinger [view email]
[v1]
Thu, 16 Jan 2025 16:38:33 UTC (1,867 KB)
[v2]
Mon, 16 Nov 2026 19:31:20 UTC (1,101 KB)
来源:arXiv cs.AI · arxiv.org