68/100

Moderate evidence

Cite
More
EmailLinkedInX

Audit experiments find AI chatbots score and pay job applicants differently by gender

Study title: Automating inequality? Gender bias in AI-mediated labor markets

Evidence type
Controlled or validated comparison
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Automating inequality? Gender bias in AI-mediated labor markets

A study in PNAS ran audit experiments in which applicant gender was randomly assigned while all other information stayed the same. Large language models produced gender-stereotypical CVs and gave different screening scores and wage recommendations depending on gender.

Read the study abstract

Generative AI is rapidly entering labor markets, with large language models (LLMs) now used to help write CVs and evaluate applicants. We study the risks this creates in a series of audit experiments in which applicant gender is randomly assigned while all other information is held constant. We find that LLMs generate gender-stereotypical CVs and exhibit gender bias in screening scores and wage recommendations. Screening scores track occupational gender segregation, penalizing women in male-dominated occupations…

The problem

In audit experiments with randomly assigned applicant gender, large language models wrote gender-stereotypical CVs, penalized women in male-dominated occupations in screening scores and favored them in female-dominated ones.

In brief

Screening scores followed the gender makeup of occupations: women were penalized in male-dominated jobs and favored in female-dominated ones. Wage recommendations followed occupational gender wage gaps when gender was stated outright in the prompt.

The authors describe two routes to the disparities, and say that because the same systems both write CV content and assess it, the routes can reinforce each other. They describe the result as measurable risks of algorithmic amplification in AI-mediated job search.

Main points

  1. Researchers used audit experiments in which applicant gender was randomly assigned and all other information was held constant.
  2. The language models generated gender-stereotypical CVs.
  3. Screening scores penalized women in male-dominated occupations and favored them in female-dominated occupations.
  4. Wage recommendations tracked occupational gender wage gaps when gender was explicit in the prompt.
  5. The wage gradient weakened when gender was signaled only through names and varied depending on which model was used.
  6. Evaluation bias means identical CV content is judged differently by gender; generation bias means gendered content the model produced affects later evaluation.

The full story

The study tested how large language models behave in two roles now common in hiring: helping write CVs and evaluating applicants. Randomly assigning gender while keeping everything else fixed let the researchers attribute differences in outcomes to gender.

Screening scores mirrored occupational gender segregation. Women received lower scores in male-dominated occupations and higher scores in female-dominated ones. Wage recommendations mirrored occupational wage gaps when gender was stated explicitly.

The authors identify two channels: evaluation bias, where the same CV content is rated differently by gender, and generation bias, where the model's own gendered CV content shapes later evaluation. Since one system can shape both CV content and its assessment, the two channels can compound.

Sources

1 on file
  1. Peer-reviewed2026-11-16

Talk it over

Three questions to start on, two more if you have time
Screening scores penalized women in male-dominated occupations and favored them in female-dominated ones. How would you weigh a favorable score in one job type against a penalty in another?

Start with a question

Audit experiments find AI chatbots score and pay job applicants differently by genderhttps://truvace.com/item/automating-inequality-gender-bias-in-ai-mediated-labor-markets

  1. Screening scores penalized women in male-dominated occupations and favored them in female-dominated ones. How would you weigh a favorable score in one job type against a penalty in another?

    Answer in the thread
  2. The wage pattern weakened when gender was signaled only through names and varied by language model. What does that tell us about how far the audit experiments reflect real hiring?

    Answer in the thread
  3. The same systems can write CVs and assess them. Who should check for generation bias and evaluation bias, and how often should they do it?

    Answer in the thread
If you have more time
  • If you were an applicant using a chatbot to write your CV, what would you check in the draft before sending it?

    Answer in the thread
  • Which occupations would you want tested next in an audit with randomly assigned gender, and why?

    Answer in the thread

What readers say