Truace
Subscribe
70/100

Moderate evidence

Cite
More
Export CSVExport JSONEmailLinkedInX

Analysis of 124 studies shows machine learning predicts suicide risk better than theories

Study title: A direct comparison of theory-driven and machine learning prediction of suicide: A meta-analysis

Evidence type
Systematic review or meta-analysis
Sources
1 sourcePeer-reviewed
Published
2 min read
Truvace record
TRV-2026-2812cite or check it

Study title: A direct comparison of theory-driven and machine learning prediction of suicide: A meta-analysis

A meta-analysis of 124 longitudinal and other studies, covering 330 effect sizes published before May 2020, compared how well four leading theories of suicide and machine learning models predict suicidal ideation, attempts and death. Machine learning models produced much higher weighted odds ratios: 13.84 for ideation, 99.01 for attempts and 17.29 for death.

Read the study abstract

Theoretically-driven models of suicide have long guided suicidology; however, an approach employing machine learning models has recently emerged in the field. Some have suggested that machine learning models yield improved prediction as compared to theoretical approaches, but to date, this has not been investigated in a systematic manner. The present work directly compares widely researched theories of suicide (i.e., BioSocial, Biological, Ideation-to-Action, and Hopelessness Theories) to machine learning models…

A direct comparison of theory-driven and machine learning prediction of suicide: A meta-analysis
"Pathologists looking into microscopes (1)" by Linda Bartlett, public domain.

The gain

In a meta-analysis of 124 studies (330 effect sizes), machine learning models predicted suicide attempts with a weighted odds ratio of 99.01, against 1.43 for theory-driven models, and also beat them on ideation (13.84 vs 2.87) and death (17.29 vs 1.08).

In brief

The theory-driven models did poorly by comparison. Their weighted odds ratios were 2.87 for ideation, 1.43 for attempts and 1.08 for death, which the authors describe as suboptimal and close to chance. Among the theories, those in the Ideation-to-Action framework did best (2.41), ahead of Hopelessness (1.83), BioSocial (1.32) and Biological (1.04).

The gap comes with caveats. Few machine learning studies reported enough data to be pooled (66 effect sizes), heterogeneity was very high, and small-sample publication bias was found in both groups. Machine learning accuracy did not improve over publication years, and very high odds ratios were the exception rather than the rule.

Main points

  1. The meta-analysis searched PubMed, PsycINFO and Google Scholar; 124 of 596 publications met inclusion criteria, yielding 330 effect sizes.
  2. Combined theory-driven models had a weighted odds ratio of 1.74, while machine learning models reached 18.09 across 66 effect sizes.
  3. Machine learning predicted attempts best (weighted odds ratio 99.01, 27 effect sizes), then death (17.29, 7 effect sizes) and ideation (13.84, 33 effect sizes).
  4. Biological theories predicted outcomes no better than chance (1.04), while Ideation-to-Action theories scored highest among theories (2.41), driven mainly by ideation.
  5. Longitudinal machine learning effect sizes (weighted odds ratio 35.82) were significantly stronger than cross-sectional ones (16.45).
  6. Machine learning accuracy did not improve across publication years (p = 0.17), while theory-driven models improved slightly (b = .023, p < .001).
  7. After correcting for publication bias with trim and fill, theory-driven models fell to 1.05 and machine learning to 17.18.

What the review found when it combined the studies

These are the review’s own pooled results, copied from its abstract. Truvace did not recalculate them.

  • Prediction of combined suicidal thoughts and behaviours by Ideation-to-Action theory

    Weighted odds ratio 2.41 (95% CI 2.21 to 2.64)

    124 studies · search to May 1, 2020

    The review’s own words
    Generally, Ideation-to-Action (wOR = 2.41, 95% CI = 2.21-2.64, k = 60) outperformed Hopelessness (wOR = 1.83, 95% CI 1.71-1.96, k = 98), Biological (wOR = 1.04; 95% CI .97-1.11, k = 100), and BioSocial (wOR = 1.32, 95% CI 1.11-1.58, k = 6) theories.

  • Prediction of suicide attempts by theory-driven models

    Weighted odds ratio 1.43 (95% CI 1.34 to 1.51)

    124 studies · search to May 1, 2020

    The review’s own words
    Theoretically-driven models demonstrated suboptimal prediction of ideation (wOR = 2.87; 95% CI, 2.65-3.09; k = 87), attempts (wOR = 1.43; 95% CI, 1.34-1.51; k = 98), and death (wOR = 1.08; 95% CI, 1.01-1.15; k = 78).

  • Prediction of combined suicidal thoughts and behaviours by machine learning models

    Weighted odds ratio 18.09 (95% CI 15.98 to 20.47)

    124 studies · search to May 1, 2020

    The review’s own words
    Effect sizes related to machine learning indicated that on the whole machine learning approaches provided superior prediction of all suicide-related outcomes ( w OR = 18.09, 95% CI 15.98–20.47 k = 66)

  • Prediction of suicide ideation by machine learning models

    Weighted odds ratio 13.84 (95% CI 11.95 to 16.03)

    124 studies · search to May 1, 2020

    The review’s own words
    Machine learning provided superior prediction of ideation (wOR = 13.84; 95% CI, 11.95-16.03; k = 33), attempts (wOR = 99.01; 95% CI, 68.10-142.54; k = 27), and death (wOR = 17.29; 95% CI, 12.85-23.27; k = 7).

  • Prediction of suicide attempts by machine learning models

    Weighted odds ratio 99.01 (95% CI 68.10 to 142.54)

    124 studies · search to May 1, 2020

    The review’s own words
    Machine learning provided superior prediction of ideation (wOR = 13.84; 95% CI, 11.95-16.03; k = 33), attempts (wOR = 99.01; 95% CI, 68.10-142.54; k = 27), and death (wOR = 17.29; 95% CI, 12.85-23.27; k = 7).

  • Prediction of combined suicidal thoughts and behaviours by Hopelessness theory

    Weighted odds ratio 1.83 (95% CI 1.71 to 1.96)

    124 studies · search to May 1, 2020

    The review’s own words
    Generally, Ideation-to-Action (wOR = 2.41, 95% CI = 2.21-2.64, k = 60) outperformed Hopelessness (wOR = 1.83, 95% CI 1.71-1.96, k = 98), Biological (wOR = 1.04; 95% CI .97-1.11, k = 100), and BioSocial (wOR = 1.32, 95% CI 1.11-1.58, k = 6) theories.

  • Prediction of suicide death by theory-driven models

    Weighted odds ratio 1.08 (95% CI 1.01 to 1.15)

    124 studies · search to May 1, 2020

    The review’s own words
    Theoretically-driven models demonstrated suboptimal prediction of ideation (wOR = 2.87; 95% CI, 2.65-3.09; k = 87), attempts (wOR = 1.43; 95% CI, 1.34-1.51; k = 98), and death (wOR = 1.08; 95% CI, 1.01-1.15; k = 78).

  • Prediction of suicide death by machine learning models

    Weighted odds ratio 17.29 (95% CI 12.85 to 23.27)

    124 studies · search to May 1, 2020

    The review’s own words
    Machine learning provided superior prediction of ideation (wOR = 13.84; 95% CI, 11.95-16.03; k = 33), attempts (wOR = 99.01; 95% CI, 68.10-142.54; k = 27), and death (wOR = 17.29; 95% CI, 12.85-23.27; k = 7).

  • Prediction of suicide ideation by theory-driven models

    Weighted odds ratio 2.87 (95% CI 2.65 to 3.09)

    124 studies · search to May 1, 2020

    The review’s own words
    Theoretically-driven models demonstrated suboptimal prediction of ideation (wOR = 2.87; 95% CI, 2.65-3.09; k = 87), attempts (wOR = 1.43; 95% CI, 1.34-1.51; k = 98), and death (wOR = 1.08; 95% CI, 1.01-1.15; k = 78).

  • Prediction of combined suicidal thoughts and behaviours by theory-driven models

    Weighted odds ratio 1.74 (95% CI 1.66 to 1.82)

    124 studies · search to May 1, 2020

    The review’s own words
    all effect sizes related to theoretically-driven models predicted combined STBs ( w OR = 1.74; 95% CI, 1.66–1.82; k = 264)

Code checks every number above against the review’s text before it is shown. The short labels are an AI model’s reading of that text, so the review’s own sentence sits under each one. A pooled figure is only as strong as the studies behind it. How Truvace uses pooled results

The full story

The authors included English-language studies; theory-driven studies had to include at least one longitudinal analysis, while machine learning studies could be cross-sectional, longitudinal or experimental because the literature was thin. Effect sizes were converted to odds ratios, pooled with random-effects models, and two coders extracted data with an intraclass correlation of .978. Of 26 additional machine learning manuscripts that lacked data, only 5 authors supplied it.

Among theories, Biological studies made up 37.8% of effect sizes, Ideation-to-Action 37.1%, Hopelessness 22.7% and BioSocial 2.3%. Ideation-to-Action predicted ideation (3.05) about twice as well as attempts (1.52), and Hopelessness predicted ideation (3.38) better than attempts (1.55) or death (1.34). The authors note that theories were built to explain causes, not to optimise prediction.

The authors caution that machine learning models were usually trained and tested on very similar samples, so a model built on adolescents may not work for veterans. They also note that high AUC values can hide poor positive predictive value, and they call for studies to report true and false positives and negatives. They add that machine learning models generally do not explain causes, which gives them less clinical use for treatment targets, and they raise privacy, consent and fairness concerns about deploying them.

Sources

1 on file
  1. PLOS ONEdoi:10.1371/journal.pone.0249833
    Peer-reviewed2021-04-12

Talk it over

Four questions to start onWho gets this benefit, and who was left out of the study or story?

Start with a question

Analysis of 124 studies shows machine learning predicts suicide risk better than theorieshttps://truvace.com/item/a-direct-comparison-of-theory-driven-and-machine-learning-prediction-of-suicide-

  1. Who gets this benefit, and who was left out of the study or story?

    The gain In a meta-analysis of 124 studies (330 effect sizes), machine learning models predicted suicide attempts with a weighted odds ratio of 99.01, against 1.43 for theory-driven models, and also beat them on ideation (13.84 vs 2.87) and death (17.29 vs 1.08).

    Answer in the thread

  2. How do we know? What can this kind of evidence show, and what can’t it?

    The evidence This rests on a systematic review or meta-analysis, from PLOS ONE.

    Answer in the thread

  3. What does it leave open? What else would you want to know before acting on it?

    The record’s own caveat Few machine learning studies supplied enough data to pool, heterogeneity was very high, and small-sample publication bias was found in both approaches, so pooled effect sizes should be read with caution.

    Answer in the thread

  4. Have you seen AI used this way where you study, work or live? Did it go the same way?

    Answer in the thread

The questions are the same on every record. The details under them are taken from this record as written.

What readers say

More from Health