Warning: Humans cannot reliably detect speech deepfakes
Speech deepfakes are artificial voices generated by machine learning models. Previous literature has highlighted deepfakes as one of the biggest security threats arising from progress in artificial intelligence due to their potential for misuse. However, studies investigating human detection capabilities are limited. We presented genuine and deepfake audio to n = 529 individuals and asked them to identify the deepfakes. We ran our experiments in English and Mandarin to understand if language affects detection pe…

In brief
Researchers played a mix of genuine and machine-generated speech to 529 participants and asked them to pick out the fakes. The test ran in English and Mandarin to see whether language changed detection accuracy or the reasoning listeners used.
The results suggest that people cannot be relied on to catch synthetic voices, which strengthens the case for technical and procedural defenses. The authors expect the task to get harder as synthesis improves, though that is a prediction rather than something this experiment measured.
Main points
- The experiment involved 529 participants who judged genuine and deepfake audio.
- Listeners correctly identified deepfakes 73% of the time.
- Detection performance did not differ between English and Mandarin.
- Giving listeners examples of speech deepfakes improved results only slightly.
The problem
People listening to genuine and AI-generated audio correctly spotted deepfakes only 73% of the time, so human hearing alone is an unreliable defense against voice-based misuse.
The rundown
The study frames speech deepfakes as artificial voices produced by machine learning models and notes prior literature that treats them as one of the largest security threats from AI progress because of their potential for misuse.
The authors say few studies had examined human detection capability. This work addressed that gap by testing listeners in two languages, English and Mandarin, and examining both accuracy and decision-making rationale.
Awareness training in the form of example deepfakes helped only marginally. The authors conclude that the difficulty of detection confirms deepfakes' misuse potential and signals a need for defenses against the threat.
Sources
- Peer-reviewedPLOS ONE2023-08-02
ace

The debate