AI models struggle to spot rare genetic diseases from patients' faces

Study title: Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study

Researchers tested three locally run, open-weight multimodal language models on 4746 facial images from the GestaltMatcher Database. Each model produced 50 ranked candidate diagnoses, and the team compared raw images, enhanced images, red-circle visual prompts and a two-stage image-to-phenotype approach.

Read the study abstract

Background: Rare genetic diseases often present with recognizable facial phenotypes. Whether multimodal large language models (MLLMs) can prioritize the diagnosis of rare genetic diseases from facial images and whether image enhancement or visual prompting can improve this process remain unclear. Objective: This study aimed to evaluate the diagnostic accuracy of MLLMs for rare genetic diseases using facial images and to assess whether super-resolution enhancement or red-circle visual prompting improved performan…

By the Truvace deskNovember 3, 20261 min read1 sourceEvidence: Moderate · 55/100 — Benchmark or lab evaluation · n = 4,746 · 1 independent sourcePeer-reviewedTRV-2026-2339· cite & verify

Diagnosing Rare Genetic Diseases From Facial Images Using Visual Prompting With Multimodal Large Language Models: Diagnostic Accuracy Study — "bars on Doctor's Hospital" by moonrat42 is licensed under CC BY-SA 2.0. To view a copy of this license, visit https://creativecommons.org/licenses/by-sa/2.0/.

In brief

Top-50 accuracy was 13.95% to 18.25% for diseases with at least six images and 0.57% to 1.35% for rarer diseases. Visual prompting helped in some models and subgroups, but the analyses suggest outputs were only weakly tied to each patient's facial features. Physician comparisons and prospective multicenter data are still needed.

Main points

The study used 4746 images: 882 from more common diseases and 3864 from rarer ones.
Top-50 accuracy was 13.95% to 18.25% on frequent diseases and 0.57% to 1.35% on rare ones.
Red-circle prompting raised numerical accuracy in 5 of 6 model-subset comparisons, and Qwen3.8-27B improved on rare diseases from 0.57% to 1.27%.
Predictions overlapped heavily after patient-image replacement, and top-1 predictions clustered on a small number of diseases.

The problem

Three open-weight multimodal AI models identified rare genetic diseases from facial images poorly, and on the rarest diseases a random baseline of 8.15% beat all three models on raw images.

The rundown

The models tested were Gemma-4-31B, Qwen3.6-35B-A3B and Qwen3.8-27B. Candidate diagnoses were mapped to Mondo Disease Ontology identifiers. The conditions compared were raw images, CodeFormer super-resolution enhancement, generated red-circle prompts, mismatched red-circle prompts, and a two-stage image-to-phenotype workflow.

In the good visual-prompt subgroup of the frequent-disease set, Gemma-4-31B improved over the super-resolution condition by 5.76 and 4.46 percentage points in the two physician-rated analyses. The image-to-phenotype workflow improved accuracy for Gemma-4-31B on frequent diseases and for Qwen3.8-27B on rare ones.

Matched red-circle inputs gave higher accuracy than mismatched inputs in all model-subset comparisons, but the adjusted P values were above .05. The authors recommend physician comparisons, prospective multicenter data and phenotype-grounded diagnostic workflows.