aceCan large language models code meaning-making? A multi-model validation of automated autobiographical reasoning scoring in Chinese adolescent narratives
Autobiographical reasoning (AR) involves drawing self-relevant meaning from personal experiences. It is central to narrative identity research, but manual coding requires substantial time and trained coders. Large language models (LLMs) have shown promise for automating other narrative coding tasks, but their capacity for meaning-making judgments that may depend on implicit event-self connections remains untested. We collected autobiographical narratives from 437 Chinese high school students and evaluated three…
In texts with moderate-to-high autobiographical reasoning, majority-vote LLM scoring reached 94.4% accuracy, suggesting partial usefulness for automated coding.
Agreement with manual coding was only moderate at best (Cohen's kappa .416), and models missed low-AR texts and over-read emotional expression, so they cannot replace formal manual coding.
The conclusion applies only to the specific models, prompts, corpus and scoring rules tested, so it may not generalize to other models, languages, or datasets.
Evidence
- Peer-reviewedBehavior Research Methods2026-10-23
How should this claim be treated?
Truvace Impact Record TRV-2026-2243, v1: “Can large language models code meaning-making? A multi-model validation of automated autobiographical reasoning scoring in Chinese adolescent narratives.” Truvace, 2026-10-24. /record/TRV-2026-2243 (accessed at citation time). sha256 75b43761abf9d4ac…
Calibration history
Every change to this record since certification, in the open. None yet — the reading has held since it entered the record.
Certified into the record
How to verify without trusting this page
Fetch the canonical text of any version from /api/record/TRV-2026-2243 and hash it yourself — for example shasum -a 256 on the saved canonical field. The result must equal content_hash, and each version’s text ends with prev:followed by the prior version’s hash (version 1 chains to 64 zeros). If a single character of any version had been altered since certification, the chain would not reproduce.