Moderate evidence
More
Can AI Stand In for Human Survey-Takers?
- Evidence type
- Benchmark or lab evaluation
- Sources
- 1 sourceJournalism
- Published
- 2 min read
- Truvace record
- TRV-2026-3055cite or check it
Methods & Innovation A quarterly digest of the Center’s latest methodological research and data science discoveries · Subscribe ↗ Preelection polling this year accurately depicted the public’s dissatisfaction with national conditions and where Americans placed the blame. It revealed that Democrats were highly motivated to vote, as well as dissatisfaction among parts of Trump’s 2024 electoral coalition. […]

The problem
When Pew Research Center replicated three American Trends Panel surveys with AI respondents, the synthetic surveys reproduced the real results poorly and were especially error-prone for Republicans and Black adults.
In brief
Pew Research Center tested whether AI can stand in for real respondents in public opinion surveys. It replicated three human surveys from its American Trends Panel using AI respondents, and the synthetic surveys did a poor job of reproducing the real results.
The errors were not spread evenly. The synthetic surveys were especially error-prone for some subgroups, including Republicans and Black adults, which matters for any pollster considering AI respondents as a cheaper substitute for people.
Pew describes the test as a response to growing interest in the method as AI becomes more advanced. The finding appears in the center's quarterly Methods & Innovation digest, alongside work on election polling accuracy and bogus respondents in online opt-in polls.
Main points
- Pew replicated three human surveys from its American Trends Panel using AI respondents instead of people.
- The synthetic surveys did a poor job of reproducing the real results.
- Errors were larger for some subgroups, including Republicans and Black adults.
- Pew said interest in AI stand-ins for survey respondents is growing as AI becomes more advanced.
- The same digest reports that preelection polls in 2026 generally understated the size of the Democratic Party's advantage.
- A separate Pew study of three ways to remove bogus respondents from online opt-in samples found that a surefire solution remains elusive.
The full story
The AI-respondent test is one item in Pew's quarterly Methods & Innovation digest. Pew built synthetic versions of three American Trends Panel surveys and compared them with the answers real panelists gave.
Pew reports that the synthetic surveys reproduced the real results poorly, with higher error for subgroups including Republicans and Black adults. Pew presents this as an assessment of using AI to stand in for public opinion respondents.
The digest also covers other threats to poll quality. Preelection polls this year captured the direction of the election but generally understated the Democratic advantage, and almost 70% of national election polls as of 2024 weighted on party affiliation or past vote choice.
Sources
1 on file- Pew Research Centerpewresearch.org
Talk it over
Who carries this cost, and what would have to change to prevent it?
Can AI Stand In for Human Survey-Takers?https://truvace.com/item/can-ai-stand-in-for-human-survey-takers
Who carries this cost, and what would have to change to prevent it?
The problem When Pew Research Center replicated three American Trends Panel surveys with AI respondents, the synthetic surveys reproduced the real results poorly and were especially error-prone for Republicans and Black adults.
Answer in the threadHow do we know? What can this kind of evidence show, and what can’t it?
The evidence This rests on a benchmark or lab test, from Pew Research Center.
Answer in the threadWhat does it leave open? What else would you want to know before acting on it?
Answer in the threadHave you seen AI used this way where you study, work or live? Did it go the same way?
Answer in the thread
The questions are the same on every record. The details under them are taken from this record as written.
ace
What readers say