Supervised 3D MRI Transformers Hit a Representation Ceiling in Brain Disease Classification
In brain MRI classification, more data usually should mean more signal, cleaner separation, and better accuracy. This preprint argues that expectation breaks down when the encoder itself compresses disease information into too few latent directions. In supervised 3D MRI transformers, the model can look powerful on paper while quietly exhausting its usable representation space long before the training set runs out.
The work focuses on brain disease classification using MRI data from the Alzheimer’s Disease Neuroimaging Initiative, with the main task set around dementia or mild cognitive impairment versus cognitively normal participants. The central problem is not simply whether a transformer can classify scans, but why its performance stops improving even as the sample size increases. The answer, in this analysis, lies in the geometry of the learned feature space.
One direction dominates. The rest barely matter.
Spectral Collapse Concentrated Disease Signal Into One Principal Component
The investigators used spectral analysis to examine how supervised vision transformers organize latent information when trained on 3D brain MRI. Their framework builds on earlier theory for simpler classifiers, where the performance curve AUC(N), trained on N samples, depends on the eigenspectrum of the encoder. In that setting, different discriminative directions begin to matter only after the sample size passes a threshold. A broad spectrum can support continued gains. A narrow one cannot.
Across the supervised transformer encoders, the disease-related representation collapsed into a single dominant principal component, with EVR@1 values between 0.90 and 0.96. In practical terms, one latent direction carried almost all of the discriminative signal. Once the encoder had concentrated information that tightly, there were no additional directions left to “turn on” as more data arrived, and the classification curve flattened.
The spike was not subtle. For the disease-trained supervised encoders, the leading eigenspectral spike was 11 to 26 times larger than predicted from the rest of the spectrum. In contrast, the frozen self-supervised DINO encoder did not show the same collapse, with a value of 0.66. The contrast matters because it suggests the plateau is not an unavoidable property of the MRI task itself; it is a property of how the supervised network chose to represent the task.
The Plateau Came From the Encoder, Not Just a Harder Classification Task
A particularly useful part of the analysis was the transfer experiment. Investigators froze each encoder and applied it to a new classification problem so that only task difficulty changed. An encoder trained on the easier dementia-versus-control task was then transferred to the harder mild cognitive impairment-versus-control setting.
If the plateau had been driven mainly by intrinsic task difficulty, one would expect the harder setting to reproduce the same failure pattern regardless of the representation. It did not. When MINiT was transferred from dementia versus CN to MCI versus CN, the threshold was 9.2, and the plateau did not reappear in the same way. But when the model was fully retrained on the harder task, the threshold jumped to 2758.5 and the plateau returned. The split points to the learned feature space, not the diagnostic label alone, as the driver of the flattening curve.
Why Supervised Training Can Box Itself In
From a mechanistic standpoint, supervised transformers are optimized to maximize class separation, and that pressure can reward compression over breadth. If the model finds one highly predictive axis, gradient updates may keep sharpening that single direction instead of preserving a richer manifold of disease-related variance. For brain MRI, where pathology can express itself across distributed atrophy patterns, vascular injury, and other structural changes, that kind of compression may be efficient in the short run and self-limiting in the long run.
The authors frame this as a form of representation collapse. The signal is not absent; it is trapped. Once a transformer pours nearly all discriminative information into one spectral spike, additional scans offer little new leverage because the encoder has already narrowed the space in which the classifier can operate. In that setting, scaling the dataset cannot fix a representation bottleneck.
This is a different failure mode from underfitting, and it is easy to miss if only final AUC is tracked. A model can look competitive early in training and still be structurally unable to benefit from more data later. That makes spectrum inspection more than a mathematical exercise. It becomes a practical audit of whether a neural encoder is leaving unused capacity on the table.
What the Spectrum Can Tell Researchers Before More Scans Are Collected
The preprint proposes a simple inference rule from the encoder’s spectrum. A large spike suggests collapse and warns that more MRI scans are unlikely to move performance much. A small spike implies the disease signal is distributed across multiple latent directions, leaving room for improvement as training data expand.
That framework could be useful for groups building medical imaging AI models for dementia and other brain disorders. Instead of waiting until a training curve flattens, researchers could inspect the eigenspectrum earlier and decide whether to change the architecture, the training objective, or the pretraining strategy. A frozen self-supervised encoder, such as DINO in this analysis, may preserve a broader feature basis than a fully supervised model trained directly on the disease label.
The clinical implication is modest but real: if a model learns too neat a shortcut, it may stop learning the biology that matters. In brain disease, where heterogeneity is the rule and not the exception, a classifier that collapses its representation space may miss the very variation needed for generalization across cohorts, scanners, and disease stages.
The work remains a preprint and has not undergone peer review. Its strongest contribution is conceptual rather than diagnostic: it offers a concrete way to distinguish a true data-limited problem from an encoder-limited one, and it ties that distinction to measurable spectral behavior rather than speculation.