How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape
Generative AI (GenAI) is a powerful technology poised to reshape Trust & Safety. While misuse by attackers is a growing concern, its defensive capacity remains underexplored. This paper examines these effects through a qualitative study with 43 Trust & Safety experts across five domains: child safety, election integrity, hate and harassment, scams, and violent extremism. Our findings characterize a landscape in which GenAI empowers both attackers and defenders. GenAI dramatically increases the scale and speed of…
Trust and Safety defenders can use generative AI to detect and mitigate harmful content at scale and support investigations and moderator wellbeing.
Generative AI increases the scale and speed of Trust and Safety attacks and lowers barriers to creating sophisticated propaganda and deepfakes.
Findings are based on a qualitative study of 43 experts and include envisioned defensive uses rather than measured deployments at scale.
Evidence
- Peer-reviewedProceedings of the 2026 CHI Conference on Human Factors in Computing Systems2026-04-13
How should this claim be treated?
Truvace Impact Record TRV-2026-0197, v1: “How Generative AI Empowers Attackers and Defenders Across the Trust & Safety Landscape.” Truvace, 2026-07-13. /record/TRV-2026-0197 (accessed at citation time). sha256 8f3f0a4e7eb1bf41…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-0197 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.
ace