Truace
Subscribe

LLMs handling misinformation for general audiences

Source article: Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation

Abstract The rapid advancement of generative artificial intelligence (AI) has introduced both opportunities and challenges in the fight against misinformation. This scoping review synthesizes recent empirical studies to explore the dual role of generative AI—particularly large language models (LLMs)—in the generation, detection, mitigation, and impact of misinformation. Analyzing 24 empirical studies, our review suggests that LLMs can generate highly convincing misinformation, often exploiting cognitive biases a…

By the Truvace desk1 min read1 sourcePeer-reviewedTRV-2026-1300· cite & verify
Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation
Electoral Integrity and Fact-Checking Workshop 2026 in Abuja 01 by Mr. Snatch. CC0 · http://creativecommons.org/publicdomain/zero/1.0/deed.en
Trace impact readingNegative state
P 73The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 68The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

A September 2025 scoping review in AI & SOCIETY synthesized 24 empirical studies on generative AI, particularly LLMs, examining how they generate, detect, mitigate, and impact misinformation.

It matters because the same systems that can convincingly produce falsehoods also show promise for detection and resistance-building, yet mitigation remains inconsistent and trust effects are documented, leaving open questions about evaluation standards, regulation, and reliable safeguards.

Main points

  1. Review synthesized 24 empirical studies on generative AI and misinformation.
  2. LLMs were found to exploit cognitive biases and ideological leanings when generating convincing misinformation.
  3. Exposure to AI-generated misinformation reduced trust and influenced decision-making.
  4. Mitigation showed mixed results with safeguards inconsistently applied.

The gain

LLMs demonstrated capacity to detect false claims and increase users' resistance to misinformation, with personalized corrections showing effectiveness.

The problem

LLMs can generate highly convincing misinformation that exploits audience biases, and exposure was found to reduce trust and influence decision-making.

The rundown

The scoping review analyzed 24 empirical studies focused on large language models across four functions: generation, detection, mitigation, and impact of misinformation.

Generation studies highlighted exploitation of cognitive biases and ideological leanings, while detection studies showed LLMs identifying false claims and boosting resistance, and mitigation studies noted personalized corrections worked but safeguards were uneven.

What this doesn’t fix

Mitigation evidence is mixed and safeguards are not consistently applied, and the field lacks standardized evaluation metrics.

Sources

  1. Peer-reviewedAI & SOCIETY2025-09-30

The debate