Truace
Subscribe

AI chatbots struggle to judge hand contracture from photos and often miss surgeon advice

Study title: Performance of Large Language Models in Image-Based Evaluation of Dupuytren Contracture

Researchers gave images of Dupuytren's contracture, a hand condition, to three AI chatbots: ChatGPT, Gemini and Claude. Each model worked in an independent session with a standardized prompt and was asked to describe the contracture and recommend treatment. Their answers were checked against the consensus of three board-certified hand surgeons.

Read the study abstract

Background Artificial intelligence (AI) has been utilized as a supplemental tool by patients and providers in the evaluation of common hand pathology. This study evaluated the ability of 3 large language model (LLM)-based AI systems (ChatGPT, Gemini, and Claude) to characterize Dupuytren's contracture involvement and recommend treatment based solely on patient images. Methods A heterogeneous set of publicly available images was obtained from open-access databases and literature sources. Images were evaluated by…

By the Truvace desk1 min read1 sourceEvidence: Moderate · 52/100 — Benchmark or lab evaluation · 1 independent sourcePeer-reviewedTRV-2026-2304· cite & verify
Performance of Large Language Models in Image-Based Evaluation of Dupuytren Contracture
Dupuytren's2010 by James Heilman, MD. CC BY-SA 3.0 · https://creativecommons.org/licenses/by-sa/3.0

In brief

The results suggest that patients who upload a photo to a chatbot may receive unreliable guidance about this hand condition. The authors say it is not yet known whether adding standardized clinical information would make the recommendations more dependable, so the study does not show how chatbots perform in realistic care settings.

Main points

  1. The chatbots tested were ChatGPT, Gemini and Claude.
  2. Images came from open-access databases and literature sources rather than real patients seen in clinic.
  3. Treatment recommendations were often within one ordinal tier of surgeon consensus.
  4. Agreement with surgeons was no greater than chance, and no consistent differences appeared among the models.

The problem

Three AI chatbots characterized Dupuytren's contracture from photos with limited accuracy, and their treatment recommendations agreed with hand surgeons no more than chance would.

The rundown

The study used a heterogeneous set of publicly available images from open-access databases and literature. Each image was evaluated in an independent stateless session with a standardized prompt, so earlier answers could not influence later ones.

Model outputs were compared with consensus assessments from three board-certified hand surgeons. The models were weak at characterizing the extent of contracture involvement, and their treatment suggestions were often close to the surgeons' tier but did not agree beyond chance.

The authors describe a gap between current model performance and surgeon consensus when judging from single patient-submitted images, and call for further research on whether adding clinical information would improve reliability.

Sources

  1. Peer-reviewedHAND2026-10-27

The debate