Truace
Subscribe

Large Language Model-Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study

Background Identifying individuals on parole or probation can help inform care pathways and ensure continuity of care in health care systems. However, this sensitive justice-related information is often absent from structured electronic health record fields and is instead documented in narrative clinical notes. Developing automated parole and probation detection (PPD) systems typically requires expert-annotated clinical data, but manual annotation is costly, time-consuming, and constrained by data sparsity and p…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-2081· cite & verify
Large Language Model-Generated Synthetic Data for Parole and Probation Detection in Clinical Notes: Development and Evaluation Study
Emoji Use in the Electronic Health Record by David A. Hanauer, Gavin C. Raab, Shira N. Hanauer, Lisa Ferguson, Kellen McClain, Guan Wang, Michelle Rozwadowski, Sung W. Choi. CC BY 4.0 · https://creativecommons.org/licenses/by/4.0

In brief

Researchers built Synth-PPD, a set of 7579 labeled clinical notes written by large language models and filtered for redundancy and quality. They used it to teach or prompt models to spot whether a patient is on parole or probation, information that is usually buried in narrative notes rather than structured health record fields.

The approach matters because manual annotation of sensitive clinical text is costly, slow, and limited by privacy concerns. The reported results come from one real-world test set of 293 Veterans Affairs notes, so how well the method works in other health systems remains untested in the supplied text.

Main points

  1. Synth-PPD contains 7579 synthetic labeled clinical notes produced with LLMs, redundancy filtering, and automated quality evaluation.
  2. Two annotators reviewed 1517 synthetic notes, and human-LLM alignment reached 90.05%.
  3. Llama 3.3-70B-Instruct with chain-of-thought prompting and Synth-PPD few-shot examples reached a macro-F1 of 87.22%, versus 80.10% zero-shot and 77.55% for the best rule-based baseline.
  4. A supervised model trained on synthetic data reached a macro-F1 of 81.43%, above the best rule-based baseline.

The gain

Training only on LLM-generated synthetic notes let models detect parole or probation status in real Veterans Affairs notes, beating rule-based baselines without expert-annotated training data.

The rundown

Identifying people on parole or probation can inform care pathways and continuity of care, but this justice-related information is often missing from structured electronic health record fields and appears only in narrative notes.

The team evaluated Synth-PPD in unsupervised and supervised settings using LLMs, traditional machine learning models, and transformer encoders. The primary outcome was macro-F1 for parole and probation detection.

Testing used 293 expert-annotated real-world clinical notes from the US Department of Veterans Affairs. The authors conclude that synthetic data can replace expert-annotated real clinical data for this task while preserving privacy.

Sources

  1. Peer-reviewedJMIR AI2026-10-19

The debate