Self-reported diagnoses proved a stronger signal than later billing codes for hidden chronic disease
In the All of Us Research Program, self-reported diagnoses carried more weight than later diagnosis codes when investigators tried to find patients with uncoded chronic fatigue, fibromyalgia, and endometriosis. The central question was not whether codes are useful—they are—but whether they are the best training label when the clinical target includes people who have already told researchers about their illness yet have not been formally coded in the record. For these underrecognized conditions, the answer tilted toward self-report.
That difference matters at the bedside and in the data pipeline. A case-finding model trained on claims or diagnosis codes can easily miss the very patients researchers most want to identify: those with symptoms, specialist encounters, or functional impairment who never receive the qualifying code at the right time, or never receive it at all.
Uncoded patients made up the larger challenge
The analysis followed survey respondents in All of Us who did not have a qualifying diagnosis code by the date of the survey and then looked for later codes. For chronic fatigue, the cohort included 149,860 respondents, among them 7,770 self-reporters, and 95% of those self-reporters had no code observed. The study design therefore separated two related but different targets: people who self-identified with the condition and people who later accumulated a diagnosis code in the record.
Investigators fit three types of models. A naive Bayes score with empirical-Bayes shrinkage was trained from released feature-by-label counts for self-reported diagnosis and from exact counts inside the data enclave for later codes. An elastic net model was also trained on participant-level data for each label. Performance was measured in a random 20% validation split, and discrimination was tested for self-report, uncoded self-reporters, and later codes.
Chronic fatigue and fibromyalgia followed the same pattern
For chronic fatigue, the summary-count score trained on self-report ranked uncoded self-reporters with an AUROC of 0.74, compared with 0.67 when the score was trained on later codes. The 95% confidence intervals were 0.72–0.75 and 0.65–0.69, respectively. When the target shifted to later codes, the two scores were nearly aligned: 0.69 versus 0.70. The signal was similar for fibromyalgia, where the self-report-trained summary score reached 0.73 for uncoded self-reporters, versus 0.64 for the code-trained score, while later-code discrimination was 0.74 versus 0.72.
Those numbers point to a simple clinical reality: self-report captures a phenotype that diagnosis codes only partially preserve. In chronic pain and fatigue syndromes, formal coding often lags behind lived disease burden. Patients may carry a diagnosis in their own account long before the administrative record reflects it, and some never receive a code that cleanly maps to the syndrome at all.
Elastic nets improved ranking of uncoded self-reporters but traded off with code-based discrimination
The participant-level elastic net models pushed performance higher for uncoded self-reporters. For chronic fatigue, the self-report-trained elastic net reached an AUROC of 0.80 versus 0.74 for the summary-count score. For fibromyalgia, the corresponding figures were 0.78 versus 0.73. Even so, the improved ranking of uncoded self-reporters came with a cost: these self-report-trained elastic nets discriminated later codes less well than code-trained elastic nets.
This split exposes a familiar tension in phenotype discovery. A model can be optimized to find the people who say they have a disease, or the people whose charts later receive a code, but those targets are not identical. Self-report appears to anchor the model closer to patient-experienced illness, while code-based training pulls it toward administrative confirmation. For chronic fatigue and fibromyalgia, the two aims overlapped enough that self-report still performed well for later codes. For endometriosis, the gap widened.
Endometriosis was the weakest and most unstable label
For endometriosis, the self-report score also ranked uncoded self-reporters above the code-trained score, 0.61 versus 0.53. Yet overall discrimination was low, and the self-report score did not discriminate later codes, where performance was 0.50 compared with 0.70 for the code-trained score. In other words, the self-report label helped identify people who endorsed the condition but did not translate into a broadly reliable predictor of later coded disease.
That pattern may reflect the clinical heterogeneity of endometriosis, variation in diagnostic delay, and the fact that confirmed endometriosis often requires a different pathway to documentation than symptom-driven self-report. The result also serves as a warning against assuming that one training label will behave the same way across conditions that share an underdiagnosis problem but differ in pathophysiology, health-seeking patterns, and coding practices.
Released summary counts performed close to participant-level modeling in the right setting
The study also compared a score fit only from released summary counts with a participant-level model. For chronic fatigue and fibromyalgia, the summary-count approach tracked uncoded self-reporters reasonably well, and in some comparisons it nearly matched later-code performance. That matters because summary-count methods can be built without direct access to individual-level records, making them attractive when data access is constrained.
The biological logic is not mysterious. If the label captures a syndrome as patients experience it, the model learns associations around symptoms, comorbidity patterns, and health-care use that may precede formal coding. If the label depends on a later billing code, the model instead learns the administrative footprint of diagnosis. In conditions where coding delay is common, those are not interchangeable signals.
Clinical validation still stands between model performance and use in practice
The investigators were direct about the limitation most relevant to translation: independent clinical validation is needed before any of these models guide case finding. Performance in a validation split inside All of Us is not the same as showing that a score can identify true clinical cases in a new health system, across different coding habits, or in populations with different access to specialty care.
The public data and code release strengthens reproducibility, and the availability of exported models may help other groups test whether self-report labels generalize beyond this cohort. Even so, the practical question remains whether a self-report-trained model can improve recruitment, chart review, or cohort assembly without importing the biases of survey response, differential diagnosis access, or selective coding. The present analysis suggests the label choice itself changes which patients rise to the top of the list. The next step is to determine whether that ranking helps clinicians and researchers find the right patients for the right reason.