Truace
Subscribe

accuracy after reviewing ChatGPT-generated answers during pediatric nephrology case-based learning in first-year medical students

Source article: Medical student reliance on artificial intelligence in nephrology education

Background Large language models such as ChatGPT are increasingly used in medical education and may influence student learning and decision-making. Despite strong performance on factual recall, limitations in clinical reasoning raise concerns about how learners engage with artificial intelligence (AI)-generated recommendations, particularly in challenging domains such as renal physiology, where foundational understanding underpins clinical application. Methods Fifty-seven first-year medical students completed 24…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-0235· cite & verify
Medical student reliance on artificial intelligence in nephrology education
"One of Ryan's classrooms. Students stay put, while teachers move from room to room." by bato93 is licensed under CC BY-SA 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/2.0/.
Trace impact readingContested
P 74The P score combines the specificity and measured human impact of the grounded problem claim with the strength of this Trace’s cited sources.

Both sides are scored from claims and sources, not community votes.

G 72The G score combines the specificity and measured human impact of the grounded gain claim with the strength of this Trace’s cited sources.

In brief

In a July 2026 peer-reviewed study, 57 first-year medical students completed 24 paired clinical and foundational questions during a pediatric nephrology and urology case-based session, answering individually, then viewing a ChatGPT-generated answer that was deliberately correct or incorrect, and re-answering.

The findings matter because they show AI assistance can produce a small net accuracy gain while also driving context-dependent reliance, especially on foundational content where students are less certain, raising questions about how to teach domain-sensitive AI literacy without undermining foundational understanding.

Main points

  1. 57 first-year medical students answered 24 paired clinical and foundational science questions in a pediatric nephrology and urology session.
  2. Students reviewed a ChatGPT-generated answer that was deliberately correct or incorrect, then re-answered, with change-to-match behavior recorded.
  3. Change-to-match occurred in 22.3% of 1357 paired responses, more frequent for foundational than clinical questions at 24.1% vs 20.4%.
  4. After adjusting for clustering and item difficulty, foundational question type had OR 1.57, 95% CI 1.08-2.29 for change-to-match.

The gain

First-year medical students showed a modest net improvement in accuracy after reviewing ChatGPT-generated answers, because incorrect-to-correct changes exceeded correct-to-incorrect changes.

The problem

First-year medical students changed answers to match ChatGPT in 22.3% of cases, with greater reliance on foundational than clinical questions, indicating context-dependent overreliance risk.

The rundown

The study enrolled 57 first-year medical students who each answered 24 multiple-choice questions pairing clinical and foundational science content in pediatric nephrology and urology, presented in random order.

After an initial answer, students were shown a ChatGPT-generated answer that was deliberately correct or incorrect, then re-answered; analyses used generalized estimating equation logistic regression to account for repeated measures and adjusted for item difficulty.

Results showed 1357 paired responses with 22.3% change-to-match, higher for foundational questions, and while incorrect-to-correct changes outnumbered correct-to-incorrect changes, the pattern demonstrates selective reliance when uncertain.

What this doesn’t fix

Findings are limited to a single case-based learning session with first-year students answering a small set of nephrology questions, constraining generalizability to other learners and settings.

Sources

  1. Peer-reviewedJournal of Nephrology2026-07-16

The debate