GPT-4o Pulled More Frailty Into View, But Prognosis Barely Moved
Frailty scores built only from structured electronic health record fields miss a familiar part of bedside medicine: the slow erosion of function that lives in physical therapy notes, narrative assessments, and other unstructured documentation. In this retrospective cohort of hospitalized adults aged 50 years or older, investigators asked whether a large language model could recover some of that missing signal and make an electronic frailty index more useful for predicting what happens after discharge. The answer was mixed. The model widened the net substantially, but it did not meaningfully sharpen overall prognostic discrimination.
The cohort included 616 hospitalizations from two studies at an academic medical center and an affiliated community hospital between February 2022 and April 2024. Mean age was 70.9 years, and just over half of hospitalizations, 329 of 616, had physical therapy documentation available for text extraction. The outcome was clinically consequential rather than abstract: a composite of discharge to a facility, readmission, or death within 90 days. More than half of hospitalizations, 320 of 616, reached that endpoint.
Physical Therapy Notes Changed Who Was Labeled Frail
The study compared two versions of a validated 35-deficit diagnosis-based electronic frailty index. The standard version relied on structured diagnosis data alone. The enhanced version used GPT-4o to extract frailty-related deficits from physical therapy documentation and fold those findings into the same frailty framework. Frailty was defined at an index threshold of 0.20 or higher.
The difference in classification was large. Standard electronic frailty assessment labeled 216 hospitalizations, or 35.1%, as frail. The LLM-enhanced version labeled 362, or 58.8%, as frail. In practical terms, the model identified many more patients as functionally vulnerable, consistent with the idea that mobility limitations, deconditioning, transfer difficulty, and assistance needs are often described in narrative form rather than captured cleanly in diagnosis codes.
One sentence captures the central tension in this paper: the model found more frailty, but more frailty did not automatically mean better prediction.
The biology behind the approach is easy to understand even when the statistics resist clean improvement. Frailty is not just a diagnosis list; it is the cumulative expression of sarcopenia, gait instability, poor reserve, dependence in activities of daily living, and the inability to recover quickly from acute illness. Physical therapists document those deficits directly, often at the bedside and often in language that structured data systems ignore. A language model can convert those descriptions into computable variables, and in this cohort that translation clearly increased capture of vulnerability. What it did not do was translate into a major gain in discrimination for the 90-day composite outcome.
Sensitivity Improved, Specificity Fell, and AUROC Barely Shifted
Both frailty tools remained independently associated with the composite outcome. The standard eFI carried an adjusted odds ratio of 1.98 with a 95% confidence interval of 1.39 to 2.83. The LLM-enhanced frailty index showed a stronger association, with an adjusted odds ratio of 2.46 and a 95% confidence interval of 1.75 to 3.47. Those estimates suggest that patients flagged as frail by either method were more likely to face death, readmission, or discharge to a facility within 90 days.
Yet discrimination changed little. The standard score produced an AUROC of 0.636, while the enhanced score reached 0.660. The difference, 0.024, had a 95% confidence interval from -0.006 to 0.053 and a p value of 0.12. In other words, the enhanced model moved the curve in the right direction, but not far enough to separate itself statistically from the simpler approach.
The tradeoff appeared immediately in test characteristics. LLM enhancement lifted sensitivity from 42.5% to 69.4%, which means far fewer high-risk patients slipped through the frailty screen. Specificity fell from 73.0% to 52.7%, so the model also broadened the frailty label to many patients who did not go on to the adverse outcome. In a hospital setting, that balance matters. A high-sensitivity frailty tool can be useful when the goal is to detect vulnerability early and trigger closer follow-up, discharge planning, or rehabilitation assessment. A lower-specificity tool can overwhelm workflows if every additional positive label demands action without a clear intervention pathway.
The study’s numbers reflect that tension with unusual clarity. The enhanced score was more inclusive, but inclusion alone does not guarantee sharper risk prediction. Frailty is a heterogeneous syndrome, and some of the deficits extracted from PT notes may identify temporary illness-related debility rather than a durable vulnerability that predicts rehospitalization or death. A patient documented as weak, unsafe for stairs, or needing assistance today may recover quickly once the acute illness resolves. The structured index may have been less sensitive, but it may also have been more tightly linked to chronic burden.
What This Means for Hospital Workflow and Risk Stratification
This work has obvious appeal for hospitals already sitting on large volumes of unstructured clinical text. If a language model can mine therapy notes for mobility impairment, dependence, and functional decline, frailty assessment could become more complete without adding new data collection burdens. The promise is operational: better detection of patients who need care coordination, discharge-to-rehab planning, or a more realistic conversation about post-acute needs.
Even so, the study also shows the ceiling of text extraction when the endpoint is broad and clinically mixed. A composite of facility discharge, readmission, and death bundles together outcomes that arise from different pathways. Some are driven by persistent frailty, some by social circumstances, and some by the acute hospitalization itself. An index that captures more vulnerability may improve identification of patients who need help, yet still fail to outperform a leaner score when the outcome reflects multiple mechanisms at once.
The population and setting also matter. This was a retrospective cohort drawn from two studies at one academic center and an affiliated community hospital, and only 53.4% of hospitalizations had physical therapy documentation available for extraction. The enhanced index therefore depended on whether those notes existed in the first place. That introduces a built-in selection issue: the patients who receive PT consults may already differ from those who do not, and the model can only read what the chart contains.
The preprint does not settle whether LLM-enhanced frailty should be used as a predictive tool or as a broader screening instrument for functional vulnerability. It does show that the two goals are not identical. For now, the clearest takeaway is narrower than a simple yes-or-no verdict: text-derived frailty assessment can uncover more patients at risk, but the jump in sensitivity does not automatically translate into substantially better prognostic performance for 90-day post-hospital outcomes.