Truace
Subscribe

Novel Approaches to Language Assessment Part 1: Artificial Intelligence-Automated Scoring of a Sentence Recall Task for School-Age Children

Purpose Artificial intelligence (AI) technologies have the potential to enhance the delivery of speech-language pathology services. One area of practice that could benefit from AI is automated screenings for developmental language disorder (DLD) in early elementary students. This report presents the elements and processes involved in evaluating an AI-based automated scoring system using archival recordings. Method A total of 947 audio samples of children completing the Redmond Sentence Recall (RSR) task, represe…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1593· cite & verify
Novel Approaches to Language Assessment Part 1: Artificial Intelligence-Automated Scoring of a Sentence Recall Task for School-Age Children
"Child speaking with Billy Bob at Showbiz Pizza in Fayetteville, Arkansas" by Ray or Jane Schumin is licensed under CC BY-SA 3.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/3.0/.

In brief

Researchers built AutoRSR, a system that uses speech recognition, natural language processing, and machine learning to transcribe children's recordings of a sentence recall task, count errors, score accuracy, and issue pass/fail screening decisions for developmental language disorder. They tested it on 947 archival audio samples from children with and without the disorder.

The results suggest that scoring of this screening task could be automated at accuracy comparable to human evaluators, which may ease demands on speech-language pathology services. The report describes an evaluation using archival recordings, and the abstract states that next steps are still to be discussed, so real-world use is not yet shown.

Main points

  1. The study used 947 archival audio samples of children completing the Redmond Sentence Recall task, covering cases with and without DLD.
  2. A subset of 130 children had Core Language scores from the Clinical Evaluation of Language Fundamentals-Fourth Edition, used as the reference standard.
  3. With Whisper speech recognition, AutoRSR reached sensitivity of .768 and specificity of .784.
  4. With Reverb speech recognition, sensitivity rose to .873 while specificity fell to .730, showing a trade-off between the two models.

The gain

An AI scoring pipeline for the Redmond Sentence Recall task matched the reference standard more closely than human-scored protocols and showed sensitivity of .768 to .873 for identifying DLD.

The rundown

The AutoRSR pipeline automates four tasks: transcribing recordings, tracking the number of errors children produce, scoring production accuracy by RSR criteria, and generating pass/fail decisions based on age-referenced expectations. It was built on automated speech recognition, natural language processing, and machine learning models.

Two speech recognition models were compared to examine trade-offs. Reverb gave higher sensitivity, meaning more children with DLD were flagged, while Whisper gave slightly higher specificity.

The system did not match human evaluators exactly. It showed some mismatches with human scorers and tended to produce lower scores, but it still converged more closely with the reference standard than human-scored protocols did.

Sources

  1. Peer-reviewedLanguage, Speech, and Hearing Services in Schools2026-10-09

The debate