68/100

Moderate evidence

Cite
More
EmailLinkedInX

Pilot study suggests AI chart summaries kept doctors' accuracy while speeding frequent users

Study title: Verifiable summarization of electronic health records using large language models to support chart review

Evidence type
Controlled or validated comparison
Sources
1 sourcePeer-reviewed
Published
2 min read

Study title: Verifiable summarization of electronic health records using large language models to support chart review

Researchers built an extract-then-abstract approach using large language models to turn longitudinal clinical notes into problem-based admission summaries for patients admitted with heart failure. The summaries were guided by disease-specific "summary bundles" that prioritize clinically relevant information.

Read the study abstract

Objectives: Information overload in electronic health records hampers clinicians' ability to efficiently extract and synthesize critical information from a patient's longitudinal health record, leading to increased cognitive burden and delays in care. This study explores the potential of large language models (LLMs) to address this challenge by generating problem-based admission summaries for patients admitted with heart failure, a leading cause of hospitalization worldwide. Materials and methods: We developed a…

The gain

In a pilot study of physicians reviewing heart failure admissions, accuracy answering clinical questions was 88.0% with LLM-generated summaries versus 86.4% with standard chart review, and frequent summary use was linked to faster completion (P = .002).

In brief

In a mixed-methods evaluation on real-world clinical notes, physicians answered patient-specific clinical questions using either the summaries or standard chart review. Accuracy was 88.0% with summaries and 86.4% without, so accuracy held steady. Access to summaries did not significantly cut overall completion time, but physicians who used them often finished faster (P = .002).

The work is an exploratory pilot, and physicians varied in how well they used the summaries and in the formats they preferred. That points to a need for customizable summaries that fit individual clinician workflows.

Main points

  1. The study targeted heart failure, a leading cause of hospitalization worldwide, using real-world longitudinal clinical notes.
  2. The method extracts relevant information first and then abstracts it, guided by disease-specific summary bundles.
  3. Physicians answered patient-specific clinical questions with the summaries and with standard chart review for comparison.
  4. Accuracy was 88.0% with summaries versus 86.4% without.
  5. Summary access alone did not significantly reduce overall questionnaire completion time.
  6. All physicians said they were likely or very likely to use the summaries in practice, and 87.5% said the summaries would save them time.
  7. Physicians differed in how effectively they used the summaries and in their preferred format.

The full story

The study addresses information overload in electronic health records, which the authors say raises clinicians' cognitive burden and can delay care. It focuses on admission summaries for heart failure patients.

The summaries come from an extract-then-abstract approach built around disease-specific summary bundles. Evaluation used real-world clinical notes and compared physicians' answers to patient-specific clinical questions using LLM summaries versus standard chart review.

Results showed accuracy of 88.0% with summaries versus 86.4% without. Overall completion time did not drop significantly with summary access, though frequent use was associated with faster completion (P = .002). Individual physicians varied in how well they used the summaries.

In self-reports, all physicians said they were likely or very likely to use the summaries clinically and 87.5% said they would save time. Format preferences varied, which the authors say supports customizable summaries and user-centered evaluation.

Sources

1 on file
  1. Peer-reviewed2026-11-13

Talk it over

Three questions to start on, two more if you have time
Accuracy was 88.0% with LLM summaries and 86.4% with standard chart review, and overall completion time did not drop significantly. How do you weigh those results against the faster work of frequent users (P = .002)?

Start with a question

Pilot study suggests AI chart summaries kept doctors' accuracy while speeding frequent usershttps://truvace.com/item/verifiable-summarization-of-electronic-health-records-using-large-language-model

  1. Accuracy was 88.0% with LLM summaries and 86.4% with standard chart review, and overall completion time did not drop significantly. How do you weigh those results against the faster work of frequent users (P = .002)?

    Answer in the thread
  2. All physicians said they were likely or very likely to use the summaries, and 87.5% said the summaries would save them time. How far can those self-reports tell us about how the summaries work in practice?

    Answer in the thread
  3. Physicians varied in how well they used the summaries and in the formats they preferred. Who should decide how customizable summaries are built for heart failure admissions, and what should they test?

    Answer in the thread
If you have more time
  • Frequent users finished faster, but were they already faster reviewers? What comparison would separate the effect of the summaries from the habits of the physicians?

    Answer in the thread
  • The summaries came from real-world longitudinal clinical notes on heart failure admissions. What would change in a hospital you know if a physician had to check a summary against the chart?

    Answer in the thread

What readers say