Truace
Subscribe

Medical student use of OpenEvidence-generated answers in a cardiovascular system course

Source article: Medical students' appraisal of curated AI answers in a cardiovascular system course

Study objective: The purpose of this study is to evaluate medical student perspective of AI-curated responses through qualitative feedback and assessment of accuracy and helpfulness. Design: First, students submitted questions about course content. Next, these questions were inputted into OpenEvidence. Finally, generated responses were edited by investigators and evaluated by students and faculty. Main outcome measures: The primary outcome of this study was the accuracy, helpfulness, and qualitative feedback for…

By the Truvace desk1 min read1 sourcePeer-reviewedTRV-2026-1503· cite & verify
Medical students' appraisal of curated AI answers in a cardiovascular system course
Shaher1 by Giardello. CC BY-SA 3.0 · https://creativecommons.org/licenses/by-sa/3.0
Trace impact readingNegative state
P 72The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 65The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

Students in a cardiovascular system course submitted questions about the material, which investigators ran through the OpenEvidence tool. The generated answers were edited by investigators and then rated by students and faculty for accuracy and helpfulness, with students also giving written feedback.

The findings suggest AI answers can serve as a useful study aid, yet students' own difficulty in verifying accuracy is a concern when learners rely on such tools. The excerpt does not give numerical ratings or faculty accuracy results, so how correct the answers actually were remains unclear.

Main points

  1. The study used questions submitted by medical students about cardiovascular course content.
  2. Questions were entered into OpenEvidence and the responses were edited by investigators before evaluation.
  3. Students and faculty both evaluated the responses for accuracy and helpfulness.
  4. Most responses were judged helpful, but students said they were often too detailed.
  5. Students reported that responses lacked beneficial visual diagrams and found accuracy hard to assess.

The gain

Students generally rated AI-curated answers to their own cardiovascular course questions as helpful.

The problem

Students had difficulty grading the accuracy of the AI answers, which were also seen as overly detailed and lacking visual explanations.

The rundown

The research followed a three-step design: collecting student questions, generating answers with OpenEvidence, and having investigators edit them before students and faculty evaluated them. The primary outcomes were accuracy, helpfulness, and qualitative feedback.

The reported results were mixed. Helpfulness ratings were generally positive, but the authors conclude that significant shortcomings exist, including excessive detail, absent visual explanations, and students' trouble in judging whether answers were accurate.

What this doesn’t fix

Responses were edited by investigators before evaluation, and the summary gives no quantitative results, so the findings may not reflect unedited AI output or measured accuracy.

Sources

  1. Peer-reviewedAmerican Heart Journal Plus: Cardiology Research and Practice2026-09-30

The debate