Pinned
Very excited to share *GSM8K-Platinum*, a revised version of the GSM8K test set!
If you’re using GSM8K, I highly recommend you switch to GSM8K-Platinum!
We built it as a drop-in replacement for the GSM8K test set. Check it out: huggingface.co/datasets/madry…
GSM8K has been a cornerstone benchmark for LLMs, but performance seemed stuck around 95%. Why?
Turns out, the benchmark itself was noisy. We fixed that, and found that it significantly affects evals.
Introducing GSM8K-Platinum!
w/@EdwardVendrow @josh_vendrow @sarameghanbeery
