EHR rules captured multiple sclerosis relapses with high sensitivity but modest precision
Electronic health records can miss the clinical story of multiple sclerosis relapse when data are scattered across admission notes, steroid orders, and neurology encounters. In a Saudi structured EHR cohort, two rule-based detection methods were tested against neurologist adjudication to see whether relapses could be identified reliably in routine care. The work matters because relapse ascertainment is not a bookkeeping exercise; in MS, a missed relapse can distort treatment assessment, inflate apparent stability, and weaken any downstream quality measurement built on EHR data.
The study drew from a large multi-regional Saudi healthcare institution and applied the algorithms to 1,812 patients with MS. Detection Method I was built around high-dose corticosteroid use plus at least one day of hospitalization. Detection Method II used a stricter admission-based pattern, requiring either a single hospitalization of at least three days or multiple consecutive neurology admissions totaling three or more days. Both approaches were then checked against neurologist-reviewed relapse episodes, with standard validation metrics calculated using 95% confidence intervals.
A total of 174 cases and 226 controls entered the final validation sample. Method I accounted for 157 cases, while Method II contributed 17. The combined performance profile was striking for sensitivity: 0.98, with a 95% confidence interval of 0.92 to 0.99. NPV was equally strong at 0.99, with a 95% confidence interval of 0.97 to 1.00. In plain clinical terms, when the rules did not flag relapse, they were usually right. When they did flag relapse, the signal was less certain.
This is the central tension in EHR phenotyping for neurologic disease. A rule set designed to avoid missed relapses will often sweep in a broader group of steroid-treated or hospitalized patients whose admissions are related to symptoms, workups, or other neurologic concerns rather than a true inflammatory relapse. The reported specificity of 0.72 and PPV of 0.50 make that tradeoff visible. Roughly half of the detected episodes represented true relapse by neurologist review, so the algorithms functioned better as sensitive screening tools than as stand-alone diagnostic arbiters.
Why corticosteroids and admission patterns matter in relapse coding
MS relapse care leaves behind a characteristic administrative footprint. High-dose corticosteroids are commonly used when clinicians judge that acute inflammatory disease activity is present, and short hospital stays or clustered neurology admissions can reflect the urgency of symptom control, imaging, and rehabilitation planning. Those markers are clinically logical proxies, which is why a rule-based approach can work at all. Still, the same footprint can appear in mimics: pseudo-relapse related to infection or heat, symptom flares that are not new inflammatory events, and admissions driven by diagnostic uncertainty rather than relapse itself.
That biological and operational overlap helps explain why the rules separated sensitivity from specificity so sharply. A broad net catches nearly all true relapses, but it also catches many episodes that look like relapse in the record without satisfying neurologist adjudication. In a real-world EHR, the code, the steroid order, and the length of stay are only partial witnesses. The clinical judgment that decides whether an event is truly a relapse remains the higher standard.
The sample composition also gives the validation result practical weight. A cohort of 1,812 MS patients from a multi-regional institution suggests the algorithm was tested in a setting where care is not confined to one clinic or one documentation style. Structured data from that kind of environment are more likely to reflect the messy reality faced by health systems trying to build registries, quality dashboards, or retrospective research cohorts. Even so, algorithm performance depends on how relapse is defined and how confidently the chart can distinguish it from closely related neurologic events.
Screening value is stronger than definitive case confirmation
The most operationally useful reading of these results is straightforward: the detection rules are suited to finding candidates for review, not replacing review. A sensitivity of 0.98 means very few true relapses were missed in this dataset, and the NPV of 0.99 suggests a negative rule result was highly reassuring. For research teams building MS cohorts, that can reduce the burden of manual chart abstraction by narrowing the list of records that deserve closer scrutiny.
The modest PPV changes the workload rather than eliminating it. Every flagged episode still needs confirmation if the goal is a clean relapse endpoint for observational studies, medication-response analyses, or utilization research. In a disease where relapse frequency can influence treatment escalation and long-term disability discussions, false positives can ripple through the data pipeline and exaggerate disease activity. A triage tool that is highly sensitive but imperfectly specific can still be valuable, provided the next step is adjudication rather than automatic acceptance.
The study also illustrates a broader point about EHR-based relapse detection in MS: administrative signals are powerful when anchored to a biologically coherent event, but they remain indirect. Corticosteroid use and hospitalization capture severity and urgency, yet they cannot fully encode neurological examination findings, lesion evolution, or the clinician’s reasoning at the bedside. That gap is where false positives enter.
What the performance profile means for MS registries and research workflows
For health systems, the immediate application is likely in registry construction and retrospective research. A high-sensitivity method can be used to flag possible relapse episodes across large datasets, then route those records for neurologist confirmation. That workflow fits the data problem well: large numbers of records, limited manual review capacity, and a need to avoid missing important events. The combination of a strong NPV and a moderate PPV argues for a staged design rather than a single-pass automated endpoint.
The study also raises a question about how relapse definitions should be operationalized across institutions. A hospitalization-based rule may work differently in systems with different admission thresholds, infusion practices, or outpatient steroid protocols. Structured data are only as portable as the care pathways they reflect. Even within one healthcare institution, coding behavior and documentation habits can shape algorithm output, which means external validation will remain essential before these methods are assumed to perform the same way elsewhere.
The bottom line is not that EHR rules can define relapse perfectly. They cannot. The result is narrower and more useful: these methods appear strong at identifying likely MS relapse episodes in real-world data, but confirmation by clinicians remains necessary before the episodes are counted as true relapse events.