Jude Fernandes (@_juderoque) / X

Really grateful to have had the opportunity to work on this project. Fairness has been an issue close to my heart and I believe our work on demographic perturbations for fairer NLP is a big step in the right direction.

Adina Williams

@adinamwilliams

May 23, 2022

We’re happy to announce our new preprint on perturbation augmentation for fairer NLP! We trained a seq2seq control-gen model to “perturb” demographic references. We use it to pretrain & finetune LMs that are fairer, without sacrificing accuracy. We measure fairness with it too 🧵

5 contributions of our research paper on perturbation augmentation illustrated with flowcharts isolating different parts of the modeling pipeline. The first is a dataset called PANDA, for Perturbation Augmentation NLP Dataset. The second is a conditional sequence to sequence model for training an automatic perturber. The third is a model called FairBERTa trained on data that had been augmented with the perturber. The fourth is a method of finetuning models on data that had been augmented with the perturber, called “fairtuning”. The final is a metric called the fairscore which measures fairness by comparing task accuracy on the original and perturber augmented evaluation datasets. There are figures describing each stage at the right.