MO-Playground¶

MO-Playground is a library for massively parallelized multi-objective reinforcement learning (MORL) for robotics. It provides GPU-accelerated multi-objective environments built on MuJoCo Playground, and two multi-objective extensions of Brax PPO that learn a whole Pareto front of policies in a single training run.

The multi-objective Cheetah with all weight on running speed (left) and on energy efficiency (right).

Concepts¶

Vector rewards. Each environment returns one reward per objective. The config file groups named reward terms into objectives, so the same environment can be trained on different objective sets without code changes.
Tradeoffs. A tradeoff is a weight vector on the probability simplex, one weight per objective. Policies are conditioned on it, and the scalarized reward tradeoff · reward drives each PPO update.

Algorithms. The algorithm: key of the config selects one of three trainers:

Algorithm	Policy	Module
`morlax`	A hypernetwork maps the tradeoff to the weights of the policy and value MLPs	`moppo.morlax`
`amor`	The tradeoff is concatenated to the observation of a single policy and value MLP	`moppo.amor`
`ppo`	Single-objective PPO on a fixed scalarization (`default_scalarization`)	`minimal_mjx`

Built on minimal_mjx. Environments, the training loop, checkpointing and rollouts come from minimal_mjx. MO-Playground adds the multi-objective layer on top.

Install¶

pip install moplayground                     # from PyPI

To train on a GPU, or to work on the code, install from source into the provided conda environment:

git clone https://github.com/dynamicmobility/moplayground.git
cd moplayground
conda env create -f environment.yml          # Linux / CUDA (mac_environment.yml on macOS)
conda activate moplayground
pip install -e ".[dev]"

Training was tested on Ubuntu 22.04 with Python 3.12 and CUDA 13. Rollouts and evaluation run on the CPU.

Quick start¶

From the repository root:

# Train MORLAX on the multi-objective Cheetah
python -m scripts.train config/mocheetah.yaml

# Roll out the trained policy at a chosen tradeoff and save a video
python -m scripts.rollout <save_dir>/<name>/config.yaml --tradeoff 1 0 0

Environments¶

`env:`	Class	Objectives in the provided config
`MOCheetah`	`MOCheetah`	running speed, energy efficiency, height
`MOHopper`	`MOHopper`	running speed, jump height
`MOWalker`	`MOWalker`	running speed, energy efficiency
`MOAnt`	`MOAnt`	x velocity, y velocity
`MOHumanoid`	`MOHumanoid`	speed, energy efficiency
`NaviGait`	`Bruce`	joint and base tracking, arm swing, rigid arms, smoothness, efficiency

Where to go next¶

Training a policy

A walk through scripts/train.py, and what a finished run directory contains.
Rolling out a policy

Roll out a trained policy at a chosen tradeoff, or a dummy zero/random controller.
Writing an environment

Subclass MultiObjectiveBase and define objectives for your own robot.
Writing an algorithm

The contract a multi-objective trainer must meet, and the building blocks provided.
API reference

Every public function and class, organized by module.

Citation¶

Janwani, Novoseller, Lawhern and Tucker. MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics. IEEE Robotics and Automation Letters, 2026. doi:10.1109/LRA.2026.3700381, arXiv:2603.09237.

@ARTICLE{moplayground,
  author={Janwani, Neil and Novoseller, Ellen and Lawhern, Vernon J. and Tucker, Maegan},
  journal={IEEE Robotics and Automation Letters},
  title={MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics},
  year={2026},
  volume={11},
  number={8},
  pages={9431-9438},
  keywords={Algorithms;International trade;Learning (artificial intelligence);Optimization;Training;Graphics processing units;Reinforcement learning;Timing;Conferences;Vectors;Reinforcement learning;machine learning for robot control;humanoid and bipedal locomotion},
  doi={10.1109/LRA.2026.3700381}}