aceDifferential Accuracy of Semantic Versus Phonological Speech Error Transcription in Aphasia: Comparing Large Language Model (Whisper) to Clinician (Speech-Language Pathologist) Performance
Purpose: Patient speech samples provide insights into cognitive and linguistic status, yet their utilization has been limited by labor-intensive manual transcription. Advances in automated speech recognition (ASR) with large language models (LLMs) potentially save time, but performance on impaired speech, such as due to poststroke aphasia, remains understudied. Method: We transcribed 4,535 trials of the Philadelphia Naming Test (PNT) administered to 26 people with aphasia (PWA) using OpenAI's Whisper speech-to-t…
Whisper transcribed a 175-trial aphasia naming test in about 2.3 minutes, versus about 180 minutes for a speech-language pathologist, and matched clinician transcripts on 92.22% of semantic errors.
Whisper matched clinician transcripts on only 23.05% of trials with phonemic errors and 75.52% overall, below the over 95% match of trained humans, with more hallucinations in more severe aphasia.
Whisper reliably transcribed semantic but not phonological errors, so automation is best suited to fluent aphasia types with fewer phonemic distortions; the sample was only 26 people with aphasia.
Evidence
- Peer-reviewedJournal of Speech, Language, and Hearing Research2026-11-02
How should this claim be treated?
Truvace Impact Record TRV-2026-2343, v1: “Differential Accuracy of Semantic Versus Phonological Speech Error Transcription in Aphasia: Comparing Large Language Model (Whisper) to Clinician (Speech-Language Pathologist) Performance.” Truvace, 2026-11-03. /record/TRV-2026-2343 (accessed at citation time). sha256 e28aa915d24b8077…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2343 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.