AI mental health chatbot use splits into distinct engagement patterns

A large real-world analysis of AI mental health chatbot use found that engagement is far more varied than a simple session count suggests. In a cohort of 102,684 users of Ash, investigators used k-means clustering across eight behavioral features to define five engagement phenotypes, then linked those patterns to short-term changes in mood, anxiety, social support, and working alliance.

The largest group was early dropouts, representing 52.2% of users. Far smaller proportions fell into power users at 1.6% and intensive users at 4.1%, while weekly users accounted for 25.3% and a newly identified concentrated user pattern made up 16.8%. The distribution alone argues against the idea that chatbot engagement behaves like a conventional app metric in which more logins automatically mean better use. Here, the shape of use varied enough to require its own clinical vocabulary.

A single statistic captures how odd, and how clinically relevant, this behavior is: 66.9% of users had at least one overnight session between 9 p.m. and 5 a.m.

Nighttime use hints at acute need rather than routine browsing

Overnight engagement is not a trivial detail. Mental health symptoms often intensify when social support is thinner, distraction is lower, and distress is harder to deflect. For a chatbot designed to provide immediate conversational support, late-night use likely reflects moments when users are seeking help outside regular care windows, when human access is limited and the barrier to entry is low. In that sense, the pattern may say as much about unmet need as it does about product preference.

The phenotype structure also suggests that engagement is multidimensional. Some users disappeared early, others returned weekly, and a smaller subset used the chatbot intensively or in a concentrated burst. Those behaviors are not interchangeable. A person who logs in once and leaves is exposing the tool to a different clinical context than someone who returns repeatedly for structured support, and the data indicate that outcome relationships shift across those patterns.

Depression improvement tracked with heavier use

Clinical change was measured in smaller subsets who completed validated scales at baseline and three weeks. Among 298 participants, depression scores on the PHQ-9 improved with an effect size of d = -0.51, anxiety scores on the GAD-7 improved with d = -0.57, and perceived social support improved modestly on the MSPSS with d = 0.22. The magnitude is not trivial for a brief digital intervention, especially in a naturalistic setting with no experimental control group reported in the abstract.

The depression signal became more informative when the investigators examined dose-response relationships. In a smaller self-reported analysis and then again in a larger model-predicted sample of 23,813 users, higher engagement aligned with larger depression improvements. Power users showed the greatest benefit, with d = -0.57, and intensive users followed closely at d = -0.50. Early dropouts improved the least, at d = -0.22. The gradient was present across the expanded sample as well, reinforcing the idea that the pattern was not a statistical artifact of the smallest cohort.

Working alliance appears to shape who benefits most

Working alliance emerged as a central clinical variable. In the sample of 11,437 users who completed the Working Alliance Inventory, stronger alliance predicted depression improvement and also moderated the relationship between engagement and social support. That finding matters because alliance is usually discussed in the context of human psychotherapy, where trust, perceived collaboration, and emotional safety influence whether users return and whether they disclose what they actually need.

In a chatbot setting, working alliance likely functions as a proxy for usability, credibility, and emotional fit. If users experience the system as coherent, responsive, and psychologically safe, they may stay engaged long enough for the intervention to exert its effects. If the interface feels flat, repetitive, or poorly matched to the user’s distress, engagement may collapse before meaningful support accumulates. The data fit that model: the strongest gains clustered among those with deeper engagement and stronger alliance, while the earliest exits showed the weakest symptom change.

The findings support a dose-response model, but not a simple one

The most clinically important message is not that more use is always better. It is that different outcomes appear to respond to different dimensions of use. Depression improved with a clear engagement gradient, yet social support and working alliance introduced additional layers that session counts alone cannot capture. A raw total of interactions misses whether the person used the chatbot repeatedly over time, concentrated use into a short interval, or dropped out after a brief trial.

That distinction has practical implications for digital mental health research and product design. If session count is treated as the main engagement measure, developers may overestimate value in some users and underestimate it in others. The phenotype approach used here gives a more nuanced picture of how people actually behave when they seek support from conversational AI. It also creates a better framework for future trials, where engagement should probably be modeled as a set of interacting behaviors rather than a single linear exposure.

Correlational evidence points to benefit, with important limits

The study provides naturalistic evidence consistent with clinical benefit from a purpose-built conversational AI tool, but the design cannot establish causality. Users were not randomized, the outcome samples were much smaller than the full user base, and the abstract does not report long-term follow-up. Those features make the work strong for describing real-world use and suggestive for outcome associations, but not definitive for proving that the chatbot caused the improvements.

Even so, the scale is hard to ignore. A dataset of 102,684 users offers a rare view into how AI mental health support is actually used outside a trial environment. The overnight usage pattern, the dominance of early dropout, the emergence of a concentrated user phenotype, and the alignment between stronger alliance and better depression outcomes together point to a field that will need more sophisticated metrics than raw activity counts. The next clinical question is not whether users click, but which patterns of use are psychologically meaningful and for whom they translate into measurable benefit.


Reporting by Dr. A. Manasa, MBBS for Medicine World Foundation, with additional data provided by Nature Medical Research.