Home
MDPSolver (mdpsolver) is a Python package for large Markov Decision Processes (MDPs) with infinite horizons. This wiki includes details about the documentation of MDPSolver and other information, such as performance tests.
- Fast solver: Our C++-based solver is substantially faster than other MDP packages available for Python, depending on the problem size and parameters. See the Performance tests section in this wiki.
- Available on PyPI.
- Two optimality criteria: Discounted and Average reward.
- Three optimization algorithms: Value iteration, Policy iteration, and Modified policy iteration.
- Three value-update methods: Standard, Gauss–Seidel, and Successive over-relaxation.
- Uses span norm or supremum norm stopping criterion depending on the selected update method.
- Supports sparse matrices and parallel computing.
- Operating systems: Linux and Windows.
Install directly from PyPI with:
pip install mdpsolver
MDPSolver works out of the box on Ubuntu 22 and newer.
Some users will encounter the version 'GLIBC_2.32' not found error when attempting to import MDPSolver in Python. In this case, it might help to compile and replace the SO-file for the optimization module in the MDPSolver package. Follow these steps to compile, install, and test the module.
Requires Visual Studio 2022 (17.9) with MSVC C++ compiler and libraries installed (see below).
After installing Visual Studio (incl. MSVC C++ compiler and libraries), install directly from PyPI with:
pip install mdpsolver
- Operating system: Windows or Linux.
- Windows users must install Visual Studio 2022 incl. the MSVC C++ compiler (see the installation instructions above).
- MDPSolver has been tested on Windows 11 and Ubuntu 22.04.5 LTS.
- Linux users must ensure that GLIBC 2.32 or newer is installed (check your version with
ldd --version).
