NiceWebRL

This repository is the official implementation of NiceWebRL: a Python library for human subject experiments with reinforcement learning environments

Table of Contents

• Install
• Working Examples
  • Paper case studies
  • More examples
• Other Jax environments compatible with NiceWebRL
• Papers that have used NiceWebRL
• Citation

NiceWebRL enables researchers to use the same environments both to train and evaluate virtual agents, and to train and evaluate humans. It supports both single-agent and multi-agent environments. As such, NiceWebRL enables AI researchers to easily compare their algorithms to human performance, cognitive scientists to test ML algorithms as theories for human cognition, and multi-agent researchers to develop algorithms for human-AI collaboration.

To enable the use of machine learning environments in online experiments, it exploits Jax—a high-performance numerical computing library—to reduce the latency from having clients communicate with a remote server. To enable easy experiment design, NiceWebRL exploits NiceGUI to enable sophisticated GUI design entirely in Python.

To facilitate its adoption, we present several working examples that researchers can use to quickly set up their own experiments.

Install

# pip install
pip install git+https://github.com/wcarvalho/nicewebrl

# more manually (first clone then)
conda create -n nicewebrl python=3.11 pip wheel -y
conda activate nicewebrl
pip install -e .

# Clone with examples (submodules)
git clone --recurse-submodules https://github.com/wcarvalho/nicewebrl
# or if already cloned:
git submodule update --init --recursive

Working Examples

Paper case studies

We present three case studies for how NiceWebRL can help researchers develop either Human-like AI, Human-compatible AI, or Human-assistive AI. The first two are from two recent papers:

• Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination

Description	Visualization	Folder
Case study 1 (Human-like AI): Developing a novel Deep RL cognitive science model with NiceWebRL [Paper]		Multitask Preplay
Case study 2 (Human-compatible AI): Developing a novel MARL algorithm for coordinating with humans with NiceWebRL [Paper]		overcooked-CEC
Case study 3 (Human-assistive AI): Developing an LLM-assistant for sequential-decision making tasks in a virtual environment.		xland-LLM-assistant

More examples

Description	Visualization	Folder
JaxMaze House maze navigation domain (single agent setting)		jaxmaze
Craftax 2D Minecraft domain (single agent setting)		craftax
XLand-Minigrid XLand-Minigrid (single agent setting)		xland-minigrid
Minigrid PPO Minigrid (single agent setting, has PPO implementation)		minigrid-ppo
Minigrid API LLM Minigrid with API-based LLM assistant		minigrid-LLM-assistant
Minigrid Local LLM Minigrid with local LLM assistant		minigrid-LLM-assistant
Dual Destination Dual Destination (Human-AI setting)		dual_destination-CEC
Overcooked Overcooked (Human-Human setting)		overcooked

Other Jax environments compatible with NiceWebRL

The following are all Jax environments which can be used with this framework to run human subject experiments. The power of using jax is that one can use the exact same environment for human subjects experiments as for developing modern machine learning algorithms (especially reinforcement learning algorithms).

When targetting normative solutions, one may want to study algorithms asymptotic behavior with a lot of data. Jax makes it cheap to do this. NiceWebRL makes it easy to compare these algorithms to human subject behavior.

Name	Domain	Visualization	Goal
Craftax	2D Minecraft		Mine and craft resources in a Minecraft-like 2D world
Kinetix	2D Physics Control		Control rigid 2D bodies to perform dynamic tasks
Navix	MiniGrid		Navigate grid environments with JAX-based MiniGrid variant
XLand–MiniGrid	XLand		Meta-RL tasks defined by dynamic rulesets
JaxMARL	Multi-agent RL		Cooperative and competitive multi-agent environments in JAX
JaxGCRL	Goal-Conditioned Robotics		Goal-conditioned control in simulated robotics tasks
Gymnax	Classic RL		Classic control, bsuite, and MinAtar environments in JAX
Jumanji	Combinatorial		From simple games to NP-hard combinatorial problems
Pgx	Board Games		Chess, Go, Shogi, and other turn-based strategy games
Brax	Physics Simulation		Differentiable physics engine for continuous control

Papers that have used NiceWebRL

• Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines
• Cross-environment Cooperation Enables Zero-shot Multi-agent Coordination
• Unsupervised Partner Design Enables Robust Ad-hoc Teamwork

Citation

If you use NiceWebRL in your work please cite it as follows:

@article{carvalho2025nicewebrl,
  title={NiceWebRL: a Python library for human subject experiments with reinforcement learning environments},
  author={Carvalho, Wilka and Goddla, Vikram and Sinha, Ishaan and Shin, Hoon and Jha, Kunal},
  journal={arXiv preprint arXiv:2508.15693},
  year={2025}
}