News & Events

The Said and the Unsaid: A Confound for Stereotype Detection in Language Models

Rachel Rudolph

Humanities Center Room D

Markedness is the pragmatic phenomenon whereby we often linguistically mark what is less expected, while leaving the default unmarked or unspecified. It is because of markedness that we find the phrase "male nurse" to be significantly more common compared to "female nurse" (this is confirmed by Google N-grams but is unsurprising to most English speaking Westerners). Markedness can be a confound for certain methods of stereotype detection in large language models (LLMs). For example, in recent work, Cheng et al (2023) found that in descriptions generated by OpenAI's GPT-3.5, the word "independent" was only significant for women personas. They recognize the counterstereotypical nature of this result but suggest that intentional bias mitigation is the source. However, markedness provides an equally likely explanation: the training data may well contain more references to "independent women" than "independent men", and not because women are stereotypically thought to be more independent than men, but rather because a man being independent tends to go without saying. In this talk, we explore the implications of markedness for stereotype detection in LLMs. We compare more direct and indirect methods of stereotype detection in order to try to tease apart associations that are due to stereotypes from ones that are due to markedness. This also prompts us to consider broader questions about the value of different methods of stereotype detection, as well as the aims of de-biasing LLMs.

This event will take place in person and via zoom. If participating online, please register in advance:

https://rochester.zoom.us/meeting/register/tJcsd-iuqD8tG9SEvQTZ9m9LcikHj4bJCYHE