Narrative and questionnaire data produced different clinical strengths in cause-of-death assignment
Cause-of-death assignment remains one of the most consequential weak points in mortality surveillance across low- and middle-income countries, where many deaths never receive a medical certificate. In South Africa, this benchmark study examined whether verbal autopsy records could do more than they currently do for automated mortality coding. The central question was straightforward: if a death record already contains both a structured symptom questionnaire and a free-text narrative, why should algorithms ignore one of those two clinical windows into the death process?
Investigators assembled 4,999 verbal autopsy records for deaths at age 12 years or older from a national sample of registered deaths in South Africa and used physician-assigned causes of death as the reference standard. The work compared unimodal pipelines that used only narratives or only structured responses with multimodal pipelines that combined the two at the data, feature, and decision levels. A locally deployed generative large language model was also tested in zero-shot fashion, without parameter updates, alongside InSilicoVA, the current automated comparator in operational use.
The structure of the study matters because cause-of-death assignment is not a single technical task. Some deaths leave crisp language in a narrative, while others are easier to classify when the questionnaire captures specific symptom patterns, duration, and exposure history. A model that can read both signals should, in theory, be less brittle than one trained on only one modality.
Supervised models beat current automated practice across the main accuracy measures
Every supervised pipeline outperformed InSilicoVA by 8.5 to 15.9 percentage points in top-1 accuracy and by 3.5 to 7.1 points in cause-specific mortality fraction accuracy. Those gains are large in a surveillance setting where even modest shifts can alter national estimates of which diseases are killing people and where resources should flow. Supervised pipelines also produced lower expected calibration error than InSilicoVA, a practical advantage because mortality systems need probabilities that are not just sharp but believable.
One sentence captures the operational consequence: the current automated comparator lagged on both individual deaths and population estimates.
Narrative-only and questionnaire-only models performed similarly in the aggregate, but they did not make the same mistakes. Of all deaths, 63.9% received identical predictions from the two unimodal pipelines, while 11.6% were correctly classified only by the narrative model and 13.6% only by the questionnaire model. The pattern is clinically intuitive. Narratives were more accurate for noncommunicable causes, where free-text descriptions often preserve course, severity, and chronic symptom patterns. Structured responses performed better for communicable causes, where specific symptom checklists can capture fever, diarrhea, respiratory symptoms, and related features more systematically.
Fusion strategy changed which causes improved and how much
The best multimodal result depended on how the two sources of information were combined. Decision-fusion pipelines added 4.2 to 5.4 percentage points in top-1 accuracy compared with the better unimodal representative, and those gains were concentrated in the most common causes. Feature fusion with cross-modal attention took a different path. It improved the less common causes instead, adding 6.6 points in balanced accuracy and 5.5 points in macro F1. In mortality coding, rare causes are the hardest to recover because they carry less training signal and are often buried inside noisy records. A mechanism that allows the model to weigh narrative and questionnaire elements jointly can pull signal out of cases that would otherwise be swallowed by class imbalance.
Feature fusion reached the highest CSMF accuracy at 95.7%. Paired differences from the better unimodal representative ranged from −0.2 to +3.4 points, and all intervals included zero. In other words, the fused systems generally improved the overall picture without showing a decisive advantage over every single unimodal comparator on every metric. That nuance is important for deployment: the best architecture depends on whether the goal is cleaner individual assignment, better recovery of population distributions, or stronger performance on underrepresented causes.
The language model excelled on rare causes, while supervised systems kept the population totals tighter
The zero-shot large language model surpassed InSilicoVA on individual-level metrics and achieved the highest balanced accuracy of any pipeline at 60.6%. Its strength on rare causes is easy to understand in clinical terms. Language models can absorb nuanced phrasing from a death narrative and may recognize uncommon symptom constellations even when they are sparsely represented in labeled training data. Yet the supervised pipelines still held the edge in population cause distribution, reaching better CSMF accuracy than the language model.
That split is not merely statistical. A health ministry needs two different kinds of confidence from a mortality system: the ability to assign the right cause to a single death, and the ability to count how many deaths belong to each cause across the country. A model that excels at one and lags at the other may still be useful, but it has to be chosen for the task it is meant to serve.
Cross-provincial testing added another layer of reassurance. The supervised gains held across all nine held-out provinces in South Africa, including every top-1 comparison and nearly every macro F1 and CSMF comparison. Generalization across provinces is a hard test because record quality, language use, care access, and local disease patterns can all shift from one region to another. Sustained performance across those held-out settings suggests the models captured more than a provincial quirk in documentation style.
Automated verbal autopsy now has a clearer role for narrative text
The most direct takeaway is that the free-text narrative should no longer be treated as decorative data. It carries independent diagnostic information and can materially improve automated cause-of-death assignment when it is incorporated into model design. The study also shows that no single fusion strategy solves every problem. Decision fusion helped common causes, feature fusion helped less common ones, and the zero-shot language model brought strength to rare causes while sacrificing some population-level fidelity.
External validation remains necessary before broad implementation across other countries, health systems, and age groups. The cohort here included deaths at age 12 years or older from South Africa, so performance in younger children and in different epidemiologic settings remains unmeasured. Even so, the benchmark is consequential: automated mortality systems can do better when they read the story of the death, not just the checklist attached to it.