Requirements:

python 3.12.9
pip install -e .

To infer on MATH500 and GSM8K, please run those two scripts

./attention_pruning_experiments/scripts/get_accuracy_data_gsm8k.sh
./attention_pruning_experiments/scripts/get_accuracy_data.sh

To reproduce our tables (coming soon, we will release our wandb experiments):

python3 attention_pruning_experiments/draw_tables.py

A substantial portion of the source code has been borrowed from the following repositories:

• https://github.com/QwenLM/Qwen2.5-Math/tree/main

If you have any problems, please open an issue in this repository or send an email to hilljun.2000@gmail.com.