跳到正文
arXiv cs.AI· Nathaniel S. Keplinger, Baiting Luo, Yunuo Zhang, Kyle Hollins Wray, Aron Laszka, Abhishek Dubey, Ayan Mukhopadhyay·· 2 小时前AI 评分34

NS-Gym:面向非平稳马尔可夫决策过程的开源综合仿真框架

NS-Gym: A Comprehensive and Open-Source Simulation Framework for Non-Stationary Markov Decision Processes

AI 导读

NS-Gym 是首个专为非平稳马尔可夫决策过程(NS-MDP)设计的仿真工具包,集成于 Gymnasium 框架内并开源。它将环境参数演化与智能体决策模块分离,提供标准化接口和基准问题,实现非平稳条件下算法的可复现评估。该成果发表于 NeurIPS 2025 Datasets and Benchmarks Track,作者还用它对多种已有算法进行了基准测试。

正文 · 原文

View PDF HTML (experimental)

Abstract:Many real-world applications require decision-making where the environmental dynamics evolve over time. These non-stationary environments pose significant challenges to traditional decision-making models, which typically assume stationary dynamics. Non-stationary Markov decision processes (NS-MDPs) offer a framework to model and solve decision problems under such changing conditions. However, there are no standardized simulation frameworks for NS-MDPs, as opposed to widely popular frameworks for stationary problems. We present NS-Gym, the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework. In NS-Gym, we segregate the evolution of the environmental parameters that characterize non-stationarity from the agent's decision-making module, allowing for modular and flexible adaptations to dynamic environments. We review prior work in this domain and present a toolkit encapsulating key problem characteristics and types in NS-MDPs. This toolkit is the first effort to develop a set of standardized interfaces and benchmark problems to enable consistent and reproducible evaluation of algorithms under non-stationary conditions. We also benchmark several algorithmic approaches from prior work on NS-MDPs using NS-Gym. We envision that NS-Gym will enable researchers to study decision-making under non-stationarity by providing standardized interfaces, simulation frameworks, and benchmark problems. Project documentation, webpage, and code can be found at this https URL and this https URL
Comments: NeurIPS 2025 Datasets and Benchmarks Track. 34 pages, 26 figures, 20 tables
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2501.09646 [cs.AI]
  (or arXiv:2501.09646v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2501.09646

arXiv-issued DOI via DataCite

Journal reference: Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track, pp. 1804-1844
Related DOI: https://doi.org/10.52202/085713-0059

DOI(s) linking to related resources

Submission history

From: Nathaniel Keplinger [view email]
[v1] Thu, 16 Jan 2025 16:38:33 UTC (1,867 KB)
[v2] Mon, 16 Nov 2026 19:31:20 UTC (1,101 KB)

来源:arXiv cs.AI · arxiv.org