Truace
Subscribe

Multimodal Visual Question Answering System for Survival Prognosis in PSMA-Targeted Radiopharmaceutical Therapy

Large language models show great promise for clinical decision support but generally lack medical imaging integration. Therefore, vision-language models have been proposed, but there is little evidence on their performance, particularly compared with conventional machine learning. Prostate-specific membrane antigen (PSMA)-targeted radiopharmaceutical therapy is a well-established treatment for metastatic castration-resistant prostate cancer (mCRPC), but it is still not known which patients will be good long-term…

By the Truvace desk2 min read1 sourcePeer-reviewedTRV-2026-1815· cite & verify
Multimodal Visual Question Answering System for Survival Prognosis in PSMA-Targeted Radiopharmaceutical Therapy
Gallium PSMA PET scan by Rauscher, Isabel; Maurer, Tobias; Fendler, Wolfgang P.; Sommer, Wieland H.; Schwaiger, Markus; Eiber, Matthias. CC BY 4.0 · https://creativecommons.org/licenses/by/4.0

In brief

Researchers built a visual question answering framework that mimics how physicians combine PSMA-directed imaging with clinical data. It uses a 3D image encoder, the RadBERT language model, and a cross-modal transformer to process baseline and interim scans plus a clinical query.

The work matters because it is still unknown which patients with metastatic castration-resistant prostate cancer will respond well to PSMA radiopharmaceutical therapy. The results are preliminary, drawn from a small retrospective sample, so prospective evaluation is still needed before clinical use.

Main points

  1. The study was retrospective and multicenter, covering 110 mCRPC patients across 2 academic institutions treated with [177Lu]Lu-PSMA I&T or [177Lu]Lu-PSMA-617.
  2. The model was trained to predict 24-month overall survival from baseline and interim (after 2 cycles) PET and SPECT images, clinical data, and a query.
  3. Internal accuracy reached 81.8%, while external validation on the PSMA-617 cohort gave 71.9% after few-shot fine-tuning.
  4. The model had the highest area under the curve (0.75) and the closest calibration to the observed survival distribution (log-rank P = 0.76).

The gain

A physician-inspired visual question answering model that integrates PSMA PET or SPECT images with clinical data predicted 24-month survival in mCRPC patients, outperforming other state-of-the-art approaches in this retrospective cohort.

The rundown

The authors note that large language models show promise for clinical decision support but generally lack imaging integration, and that evidence on vision-language models versus conventional machine learning is thin. This study tested that gap in a prostate cancer therapy setting.

Performance was compared with state-of-the-art approaches, and accuracy was comparable to the query-free attention multimodal fusion model. The authors describe the prognostic improvement as slight yet comparable overall.

External validation required few-shot fine-tuning, and the authors conclude that VQA approaches warrant future evaluation rather than immediate adoption. They say the framework showed adaptability across different PSMA ligands.

Sources

  1. Peer-reviewedJournal of Nuclear Medicine2026-10-15

The debate