Valentin Hofmann (@vjhofmann) / X

Valentin Hofmann

377 posts

Valentin Hofmann

@vjhofmann

Assistant Professor @CisLMU @LMU_Muenchen | Formerly @allen_ai @uwnlp @UniofOxford @stanfordnlp @GoogleDeepMind

valentinhofmann.github.io

Joined February 2020

Pinned
Valentin Hofmann
@vjhofmann
Mar 4, 2024
💥 New paper 💥 We discover a form of covert racism in LLMs that is triggered by dialect features alone, with massive harms for affected groups. For example, GPT-4 is more likely to suggest that defendants be sentenced to death when they speak African American English. 🧵
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
We find that the covert, raciolinguistic stereotypes about speakers of African American English embodied by LLMs are more negative than any human stereotypes about African Americans ever experimentally recorded, although closest to the ones from before the civil rights movement.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Prior work has focused on racial bias displayed by LLMs when they are prompted with overt mentions of race. By contrast, racism in the form of dialect prejudice is completely covert since the race of speakers is never explicitly revealed to the models.
Valentin Hofmann
@vjhofmann
Oct 18, 2023
📢 Life update 📢 I'm thrilled to share that I've started as a Young Investigator (postdoc) at @allen_ai! I'll be working with @nlpnoah and @HannaHajishirzi in the @ai2_allennlp team. Main source of distraction so far: seaplanes! 🛩️
Valentin Hofmann
@vjhofmann
Jun 3, 2021
BERT's tokenizations are sometimes morphologically superb-iza-rre, but does that impact BERT's semantic representations of complex words? Our upcoming #ACL2021NLP paper (w/ Janet Pierrehumbert & @HinrichSchuetze) takes a look at this question. arxiv.org/pdf/2101.00403… /1
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Second, when LLMs are asked to pass judgment on defendants who committed murder, they choose the death penalty more often when the defendants speak African American English rather than Standardized American English, again without being overtly told that they are African American.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
We analyze dialect prejudice in LLMs using Matched Guise Probing: we embed African American English and Standardized American English texts in prompts that ask for properties of the speakers who have uttered the texts, and compare the model predictions for the two types of input.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Finally, we show that current practices of human feedback training do not mitigate the dialect prejudice, but can exacerbate the discrepancy between covert and overt stereotypes, by teaching LLMs to superficially conceal the racism that they maintain on a deeper level.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Our results point to two risks: that users mistake decreasing levels of overt prejudice for a sign that racism in LLMs has been solved, when LLMs are in fact reaching increasing levels of covert prejudice.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
How does model size affect dialect prejudice? We find that increasing scale does make models better at understanding African American English and at avoiding prejudice against overt mentions of African Americans, but perniciously it makes them *more* covertly prejudiced.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
First, our experiments show that LLMs assign significantly less prestigious jobs to speakers of African American English compared to speakers of Standardized American English, even though they are not overtly told that the speakers are African American.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Crucially, the stereotypes that LLMs display when they are overtly asked about their attitudes towards African Americans are more positive in sentiment, and more aligned with stereotypes reported in surveys today (which are much more favorable than a century ago).
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
What is it specifically about African American English texts that evokes dialect prejudice in LLMs? We show that the covert stereotypes are directly linked to individual linguistic features of African American English, such as the use of "finna" as a future marker.
Valentin Hofmann
@vjhofmann
Mar 4, 2024
Replying to @vjhofmann
Please check out our paper for many more experiments and an in-depth discussion of the wider implications of dialect prejudice in LLMs: arxiv.org/abs/2403.00742 You can find all our code on GitHub: github.com/valentinhofman…