Truace
Subscribe

Mixing patient notes with structured data in AI beat single-source models by 6.4% on average

Study title: Large language models for multimodal clinical predictive modeling: a systematic review

Researchers screened 641 studies on predictive modeling with electronic health records and kept 57 that used large language models to bring unstructured clinical text together with structured patient data. They sorted the methods into embedding, extraction, and text fusion approaches.

Read the study abstract

The electronic health record (EHR) comprises structured and unstructured data, both of which are integral for comprehensively representing patient history. The emergence of large language models (LLMs) has enabled the incorporation of unstructured data modalities into machine learning models that leverage EHR data. In this systematic review, we characterize studies that utilize LLM-enabled multimodal predictive modeling with EHR data. Following a screening of 641 studies, 57 met inclusion criteria. We identified…

By the Truvace desk1 min read1 sourcePeer-reviewedTRV-2026-2237· cite & verify
Peer-reviewed study: npj Digital Medicine

In brief

The result suggests that clinical notes, which hold much of a patient's history, can add predictive value when language models are used to handle them. The review frames this as potential rather than proof of better care, and it flags sample size, interpretability, and computational cost as open considerations for future tools.

Main points

  1. The review screened 641 studies and included 57 that met its criteria.
  2. Three integration strategies were identified: embedding, extraction, and text fusion.
  3. The strategies are distinguished by the format of unstructured data at the time of fusion.
  4. Average performance rose 6.4% over single-modality models.
  5. Sample size, interpretability, and computational costs were named as considerations.

The gain

Across 57 included studies, combining unstructured EHR text with structured data through LLM-based strategies outperformed single-modality models by an average of 6.4%.

The rundown

The EHR combines structured data with unstructured data, and both are described as integral for representing a patient's history. LLMs have made it possible to incorporate the unstructured modalities into machine learning models that use EHR data.

LLMs supported modeling in three ways: generating note embeddings for downstream prediction, extracting clinically informative variables, and summarizing long text documents before integration.

The authors say the work supports the potential value of LLMs for EHR-based predictive modeling and offers a framework for future multimodal modeling tools.

Sources

  1. Peer-reviewednpj Digital Medicine2026-10-22

The debate