submitit

submitit submits Python functions to Slurm from Python, and returns their results, errors and logs. The same code runs locally.

pip install submitit  # or: conda install -c conda-forge submitit

Requires Python 3.11+.

import submitit


def add(a, b):
    return a + b


executor = submitit.AutoExecutor(folder="logs/%j")
executor.update_parameters(timeout_min=10, slurm_partition="<your_partition>")
job = executor.submit(add, 5, 7)
print(job.job_id)
assert job.result() == 12

Documentation: https://facebookincubator.github.io/submitit/

By chronological order: Jérémy Rapin, Louis Martin, Lowik Chanussot, Lucas Hosseini, Fabio Petroni, Francisco Massa, Guillaume Wenzek, Thibaut Lavril, Vinayak Tantia, Andrea Vedaldi, Max Nickel, Quentin Duval, Rushil Patel. Contributions are welcome.

Submitit is released under the MIT License.