Detecting Music Performance Errors with Transformers
AAAI 2025

Polytune introduces a transformer-based model for end-to-end music performance error detection.

Read the Paper

Click the thumbnail to watch the demo on YouTube.

1. Install Python 3.11

2. Create and activate the Conda environment:

   conda env create -n polytune python=3.11
   conda activate polytune

3. Install required packages:

   pip install -r requirements.txt

python train_polytune.py \
  --config-path="config" \
  --config-name="config_maestro/coco" \
  'devices=[0]' \
  'hydra/job_logging=disabled' \
  'model="polytune"' \
  'dataset="MAESTRO/CocoChorales"' \
  'split_frame_length=2000'

python test_polytune.py \
  --config-dir="config" \
  --config-name="config_maestro/coco" \
  model="polytune" \
  path="pretrained.ckpt" \
  eval.eval_dataset="MAESTRO/CocoChorales" \
  eval.exp_tag_name="Polytune" \
  hydra/job_logging=disabled \
  eval.is_sanity_check=True \
  eval.contiguous_inference=True \
  split_frame_length=2000

python polytune_test_inference.py \
  --config-dir="config" \
  --config-name="config_maestro" \
  model="polytune" \
  path="pretrained.ckpt" \
  hydra/job_logging=disabled \
  eval.is_sanity_check=True \
  eval.contiguous_inference=True \
  split_frame_length=2000

The model produces a multi-track MIDI file:

• Track 1: Extra notes
• Track 2: Missing notes
• Track 3: Correct notes

• CocoChorales-E: Download
• MAESTRO-E: Download

Download requires Globus or Globus Connect Personal.

To download:

1. Log in and follow setup instructions
2. Visit the dataset link
3. Click "Transfer or Sync to..."
4. Select your local machine or HPC endpoint

You can also generate your own error datasets using our code:
CocoChorales-E_MAESTRO-E GitHub Repo

Pretrained checkpoints are available here:
Download from Hugging Face

The baseline implementation (MT3) can be found in:
./baseline/MT3_baseline/

There are a few unused label files currently included in the CocoChorales dataset. The data loader automatically filters them.

If you use Polytune or our datasets, please cite:

@inproceedings{chou_detecting_2025,
  author    = {Chou, Benjamin Shiue-Hal and Jajal, Purvish and Eliopoulos, Nicholas John 
               and Nadolsky, Tim and Yang, Cheng-Yun and Ravi, Nikita and Davis, James C. 
               and Yun, Kristen Yeon-Ji and Lu, Yung-Hsiang},
  title     = {Detecting Music Performance Errors with Transformers},
  booktitle = {AAAI Conference on Artificial Intelligence},
  publisher = {AAAI},
  year      = {2025}
}