Army health assessments identified a small high-risk group for suicide-related outcomes
Periodic health assessments already collect a stream of administrative and clinical information from active-duty soldiers. This study asked whether that routine material could do more than document readiness and deployment status — whether it could also identify the subgroup most likely to die by suicide or make a nonfatal suicide attempt over the next 24 months.
The answer was yes, but only within a narrow slice of the force. The predictive models separated soldiers at materially elevated risk from everyone else, yet most events still occurred outside the highest-risk fraction. In suicide prevention, that is an unforgiving pattern: a tool can be statistically strong and still miss much of the outcome burden when the event itself is rare.
Researchers developed machine learning models using periodic health assessment data from regular U.S. Army soldiers completed between 2015 and 2019. Separate models were built for suicide and for suicide attempt, using 70% training samples and a 30% test sample to evaluate performance. For suicide, all cases were included in training along with a stratified control sample; for attempts, the same approach was used, with an added adjustment for loss to follow-up because soldiers may leave service before the 24-month horizon ends.
Rare outcomes create a difficult target for prediction
The cohort was large: PHA data were available from 668,684 regular U.S. Army soldiers, and 1,932,243 assessments were included in the analysis. Most assessments came from male soldiers, and the sample included substantial representation from soldiers aged 28 years or older and from married soldiers. Those details matter because the model had to work inside a real operational system, not in a narrowly defined psychiatric clinic.
Over 24 months, adjusted suicide prevalence was 66.7 per 100,000, while adjusted suicide attempt prevalence reached 704.1 per 100,000. Those rates frame the core statistical challenge. Even a well-performing algorithm must navigate a base rate so low that false positives can swamp the actual cases if the threshold is set too loosely. A suicide prediction tool in this setting is less like a broad screening test and more like a triage instrument designed to concentrate attention where it is most needed.
Test-sample discrimination for suicide reached an AUROC of 0.72, with an Integrated Calibration Index of 0.0003 and a Brier score of 0.0007. For suicide attempts, the model performed better, with an AUROC of 0.81, an Integrated Calibration Index of 0.0014, and a Brier score of 0.0070. Calibration was strong for both outcomes, meaning predicted probabilities tracked closely with observed risk rather than drifting far above or below reality.
High discrimination did not translate into broad case capture
The sharpest result came from the way risk concentrated at the top of the score distribution. Suicide prevalence was meaningfully elevated only in the highest-risk 5% of PHAs, with sensitivity of 18.6%. Suicide attempt risk was elevated only in the highest-risk 10% of PHAs, with sensitivity of 46.5%. In other words, the model could enrich the risk pool, but only by focusing on a relatively small segment of assessments.
That pattern is exactly what makes suicide prevention operationally difficult. A preventive intervention can be efficient only if it is aimed where the yield is highest, yet the clinicians and commanders using the model must accept that many future events will still arise among soldiers who do not land in the top decile or top percentile of predicted risk. The model is useful for targeting, not for declaring safety elsewhere.
The study’s design fits that reality. Investigators were not trying to replace clinical judgment or create a universal diagnostic screen. They were trying to identify who might be eligible for a remote group dialectical behavior therapy-based skills training intervention, a preventive strategy that can only be deployed effectively if the intake process identifies a manageable number of soldiers with elevated risk.
Administrative data can support prevention, but only within strict limits
The appeal of this approach is obvious. Periodic health assessments are already embedded in Army life, so using existing data avoids adding a new burden to soldiers or clinicians. Administrative and health-system variables also carry a different kind of signal than self-report alone, especially in a setting where stigma, fatigue, or command concerns may suppress disclosure.
Still, the model’s reach was bounded by the data available at the time of assessment. Administrative records can capture prior care, service history, and other correlates of risk, but they do not fully represent the psychological dynamics that precede suicidal behavior. Acute hopelessness, agitation, relationship rupture, substance use escalation, and access to lethal means often develop faster than annual screening cycles can track. A 24-month risk horizon is useful for matching the rhythm of annual assessments, yet suicidal crises can escalate in days or hours, not months.
The distinction between suicide and attempt models also matters clinically. Attempts were more common and therefore easier to predict with better discrimination, while suicide remained much rarer and harder to capture precisely. That difference likely reflects both event frequency and the biological and behavioral heterogeneity of completed suicide, where lethality, intent, means, and opportunity can diverge sharply across cases.
A separate finding linked the predicted probabilities for suicide and attempt only modestly, with a Pearson correlation of 0.29. Risk for death by suicide and risk for nonfatal attempt overlap, but they are not interchangeable. A soldier with a high probability of one outcome is not automatically a high-probability case for the other, and any implementation strategy will need to preserve that separation rather than collapse it into a single risk label.
The practical ceiling of machine learning in military suicide prevention
The study shows that machine learning can work inside the Army’s routine assessment infrastructure, producing calibrated predictions from data already in hand. It also shows the ceiling. Even with strong statistical performance, the model concentrated elevated suicide risk in only the top 5% of assessments and elevated attempt risk in the top 10%, leaving the majority of future cases outside the highest-risk strata.
For prevention planning, that means the model is best understood as a selective gateway to intervention rather than a comprehensive solution. Its value lies in directing scarce clinical resources toward soldiers most likely to benefit from additional assessment, outreach, and skills-based treatment. Its limitation lies in the same place as every other suicide prediction effort: the event is uncommon, the pathways are diverse, and the moment of crisis can outrun the cadence of routine surveillance.
The study’s 24-month framework, its reliance on administrative PHA data, and the use of Army records and the National Death Index define both its strength and its boundary. The next clinical question is not whether prediction can be improved in the abstract, but which added data elements, assessment intervals, and intervention pathways can turn risk stratification into fewer deaths and fewer attempts in a force that cannot wait for perfect prediction.