The primary purpose of the torchforge ecosystem is to separate infra concerns from model concerns thereby making RL experimentation easier. torchforge delivers this by providing clear RL abstractions and one scalable implementation of these abstractions. When you need fine-grained control over placement, fault handling/redirecting training loads during a run, or communication patterns, the primitives are there. When you don’t, you can focus purely on your RL algorithm.

Key features:

• Usability for rapid research (isolating the RL loop from infrastructure)
• Hackability for power users (all parts of the RL loop can be easily modified without interacting with infrastructure)
• Scalability (ability to shift between async and synchronous training and across thousands of GPUs)

⚠️ Early Development Warning torchforge is currently in an experimental stage. You should expect bugs, incomplete features, and APIs that may change in future versions. The project welcomes bugfixes, but to make sure things are well coordinated you should discuss any significant change before starting the work. It's recommended that you signal your intention to contribute in the issue tracker, either by filing a new issue or by claiming an existing one.

View torchforge's hosted documentation: https://meta-pytorch.org/torchforge.

You can also find our notebook tutorials (coming soon)

torchforge requires PyTorch 2.9.0 with Monarch, vLLM, and torchtitan.

Install torchforge with:

conda create -n forge python=3.12
conda activate forge
./scripts/install.sh

ROCm users can install with the dedicated script:

conda create -n forge python=3.12
conda activate forge
./scripts/install_rocm.sh

Notes:

• PYTORCH_ROCM_ARCH is auto-detected when possible; set it manually if detection fails (example: PYTORCH_ROCM_ARCH=gfx942).
• ROCM_VERSION is auto-detected when possible; set it manually if detection fails (example: ROCM_VERSION=6.4).
• For ROCm 7.x, PyTorch stable 2.9.0 is not available. The script defaults to nightly wheels. You can override with PYTORCH_CHANNEL=stable|nightly.
• ROCm builds install Monarch with USE_TENSOR_ENGINE=0, so RDMA and distributed tensor features are disabled for now.
• Optional flags: --use-sudo (system packages) and --skip-amdsmi (skip amdsmi install).

The install script installs system dependencies along with torchforge. Note that this install script uses DNF, but could be easily extended to other Linux OS.

Optional: By default, the packages installation uses conda. If you want to install system packages on the target machine instead of conda, you can pass the --use-sudo flag to the installation script: ./scripts/install.sh --use-sudo.

Pixi combines benefits of uv with access to conda forge for system dependencies. pixi.toml provides a manifest with build tasks with install as a the combined install all task.

Install pixi:

curl -fsSL https://pixi.sh/install.sh | bash

Install torchforge with pixi:

pixi run install

Note: We are actively working on enabling pure uv installation. Currently, Conda is the recommended approach. uv support is not fully working at the moment but is being tracked in issue #494.

After install, you can run the following command and should see output confirming GRPO training is running (you need a minimum 3 GPU devices):

python -m apps.grpo.main --config apps/grpo/qwen3_1_7b.yaml

To run SFT on a Llama3 8B model, run

python -m apps.sft.main --config apps/sft/llama3_8b.yaml

Source code is made available under a BSD 3 license, however you may have other legal obligations that govern your use of other content linked in this repository, such as the license or terms of service for third-party data and models.