Reinforcement learning targets the minutes and hours when suicidal ideation changes
Suicidal ideation does not sit still long enough for many standard monitoring schedules to catch it. ARES-SI, short for adaptive reinforcement-enhanced sampling for suicidal ideation, was built around that problem: it personalizes ecological momentary assessment timing so the next prompt lands when risk is most informative, not merely when a fixed protocol says it should.
The system combines two prediction tasks with a decision engine. Random forest models estimate the probability of active suicidal ideation and the likelihood of non-response, then a Q-learning algorithm chooses among three assessment intervals: 30 minutes for high-risk states, 3 hours for low-risk states, and the next day when burden should be reduced. In practical terms, the design tries to preserve signal during danger periods while avoiding excessive prompts when the momentary risk profile is low.
The benchmark was not a theoretical simulation alone. Investigators trained the system using ecological momentary assessment data from 98 participants drawn from three studies with sampling frequencies ranging from 4 to 15 assessments per day, producing 2,359 positive active suicidal ideation responses across 28-day protocols. Validation testing included 34 participants and 374 positive active suicidal ideation responses. In that smaller held-out sample, the random forest models reached an AUC of 0.865 for active suicidal ideation and 0.750 for non-response, strong enough to support a downstream scheduling model built on those predictions.
Why timing matters in suicidal ideation monitoring
Fixed-schedule monitoring assumes risk rises and falls on a uniform clock. Suicidal thinking usually does not. It can intensify after interpersonal conflict, worsen across a stressful afternoon, or ease by the next morning, which means a prompt sent too early or too late may miss the very window clinicians need to observe. A system that adjusts to the patient’s recent pattern of responding and endorsing ideation aims to capture the volatility itself, not just the average state.
The performance gains were substantial. ARES-SI showed 65% higher action selection accuracy than random scheduling, with accuracy of 0.781 versus 0.474 and macro F1 of 0.650 versus 0.377. Those gains are not cosmetic. In a high-stakes monitoring environment, the difference between a prompt delivered into a meaningful risk window and one sent into a quiet interval can determine whether the data collected are clinically useful or merely burdensome.
The model also improved detection of the most concerning moments. For 30-minute high-risk scheduling, recall reached 0.810 compared with 0.392 under random scheduling. On the low-risk side, 3-hour precision reached 0.972 versus 0.876. Read together, those numbers suggest the system did two things at once: it found the short, dangerous windows more reliably and still avoided flooding participants with unnecessary assessments when the probability of near-term escalation was lower.
What the validation metrics say about digital suicide risk monitoring
The model’s strength comes from pairing risk prediction with burden management. A pure classifier can tell a team whether suicidal ideation is likely; it does not decide when the next question should be asked. Q-learning introduces a sequential decision framework, which is better aligned with a fluctuating symptom than a one-off estimate. For suicidal ideation, where risk can shift quickly and the cost of missed detection is high, that extra layer is clinically meaningful.
The sample size remains modest, and the work stays within the boundaries of a validation study. Still, the training set included multiple cohorts with varying assessment frequency, which gives the approach some resilience against overfitting to a single study design. The validation set also carried a substantial number of positive active suicidal ideation responses relative to the sample size, helping test whether the scheduler could keep up when ideation actually appeared rather than only when risk was assumed.
One striking feature is the separation between the tasks. Predicting active suicidal ideation is one problem; predicting whether a participant will answer a prompt is another. In digital mental health, non-response is not a trivial nuisance variable. It can reflect fatigue, disengagement, or a moment of acute distress in which the participant is least likely to answer. By modeling both processes, ARES-SI tries to make the schedule sensitive to real-world data loss as well as real-world risk.
A step toward individualized assessment, not a replacement for clinical care
This work points toward a more adaptive form of suicide risk surveillance, one that uses prior response patterns and momentary state to decide when to ask the next question. That could be useful in research protocols, smartphone-based monitoring programs, and potentially stepped-care systems where clinicians need richer time-stamped data on symptom movement. The immediate bedside consequence is straightforward: fewer missed high-risk moments and less wasted participant effort when risk is low.
The biological logic behind the approach is behavioral rather than pharmacologic. Suicidal ideation is dynamic because stress exposure, affective reactivity, sleep disruption, and social context do not stay fixed. Sampling has to move with those shifts if the goal is to observe the symptom at the right temporal resolution. Reinforcement learning is a good fit for that job because it can optimize decisions over time instead of treating every assessment as independent.
Funding came from the National Institute of Mental Health under R21MH131978. The article also acknowledges support from the Center for Research Computing at the University of Notre Dame for maintaining the ecological momentary assessment delivery system used in the study. No competing interests were declared.
The main limitation is the same one that shadows most early digital psychiatry tools: validation in a controlled dataset is not the same as deployment in a messy clinical setting, where adherence, safety workflows, and escalation pathways vary widely. A larger prospective implementation study will need to show whether the timing advantage persists outside the development datasets and whether the system remains reliable across different populations, settings, and levels of clinical severity.