AI-Generated Stroke Driving Leaflets Fell Short of the Gold Standard
Patient advice on driving after stroke has to do more than read well on a page. It has to give people something they can act on safely, without missing the regulatory details that decide whether they can drive, when they must stop, and when specialist review becomes relevant. In a 2025 Cureus study, eight leaflets on driving after stroke were put under formal scrutiny: one from the Stroke Association and seven drafted by leading large language models, including ChatGPT-5, ChatGPT-5o, Microsoft Copilot, Claude Sonnet 4, Gemini 2.5 Flash, Perplexity AI, and DeepSeek v3.1.
The comparison was not casual. Each leaflet was converted to plain text and scored with validated tools covering readability, reliability, understandability, actionability, misinformation, and referencing quality. A weighted composite score adapted from a prior ophthalmology study was then used to compare the materials in a single framework. The result was blunt: the Stroke Association leaflet remained the benchmark.
The Stroke Association Leaflet Kept the Highest Composite Score
The Stroke Association leaflet scored 72.8% on the weighted composite, the highest in the set. DeepSeek v3.1 followed at 68.3%, while Microsoft Copilot reached 66.4% and ChatGPT-5 (Thinking) came in close behind at 66.3%. The remaining AI-generated leaflets scored lower, with Gemini 2.5 Flash, ChatGPT-5o (Standard), Claude Sonnet 4, and Perplexity AI spanning 29.8% to 59.1%.
No model overtook the expert-authored leaflet. That matters because the subject is not cosmetic prose or generic health advice; driving after stroke carries legal, safety, and public-health consequences. A leaflet that misses seizure guidance or transient ischemic attack information can create confusion at the exact moment a patient needs certainty.
Readability Improved, but Accuracy Still Drove the Ranking
Readability across all eight leaflets sat in a secondary-school range, with a median Flesch-Kincaid Grade level of 8.5. On paper, that is encouraging. It means the AI-generated leaflets generally produced text that was easier to process than much of the dense patient information patients encounter in routine clinical care. Yet readability alone did not translate into trustworthy guidance.
The Stroke Association leaflet delivered perfect factual accuracy and perfect referencing. In contrast, Microsoft Copilot and DeepSeek showed occasional omissions in guidance related to seizures and transient ischemic attack. Those gaps are not trivial. Seizure restrictions and TIA-related advice sit close to the clinical and legal thresholds that determine whether a person can safely return to the road. A leaflet can be fluent and still fail the patient if it leaves out the decisive rule.
The pattern across the comparison was consistent: AI systems could generate cleaner prose, but the human-authored leaflet remained stronger where precision mattered most. Better sentence structure did not rescue weaker sourcing. Better phrasing did not close the gap on factual completeness.
Why Stroke Driving Advice Is a Hard Test for Language Models
Driving after stroke sits at the intersection of neurology, rehabilitation, and regulation. The content has to communicate functional risk, time-based restrictions, and exceptions with enough precision that a patient can act without second-guessing. Large language models are built to produce coherent language, but coherence is not the same as clinical reliability. When the topic is a medication list or a benign self-care tip, small omissions may be inconvenient. When the topic is legal fitness to drive after stroke, omissions can change behavior in ways that affect both the individual and others on the road.
The study’s results fit that reality. AI systems approached professional standards in readability and, in some cases, structure. DeepSeek v3.1 stood out as the closest overall challenger, with near-comparable readability and organization. Microsoft Copilot and ChatGPT-5 also produced comparatively strong materials. Even so, the expert leaflet retained an edge because it combined readable language with complete sourcing and accurate content.
What the Composite Score Captured and What It Did Not
The investigation used a multi-domain approach rather than a single readability index. That was important because patient leaflets fail in different ways. One leaflet may be easy to read but unreliable. Another may be factually dense but inaccessible. The study’s scoring structure included DISCERN for reliability, PEMAT-P for understandability and actionability, misinformation scoring, and referencing quality, alongside readability metrics such as Flesch-Kincaid Grade, Flesch Reading Ease, Automated Readability Index, and Gunning Fog Index.
Across those dimensions, the stroke leaflet from the Stroke Association emerged as the most balanced document. It was not only readable; it was dependable, actionable, and properly referenced. The AI-generated leaflets were more uneven. Some approached the target on readability and usability, yet still lost ground through weaker source transparency and occasional factual gaps. In practical terms, the models performed better as drafting assistants than as autonomous authors.
Clinician and Legal Oversight Remains the Safest Path
The study points toward a hybrid workflow rather than full automation. AI can accelerate the first draft, simplify wording, and improve the layout of patient-facing information. It can help transform specialist language into text that more patients can understand. But the final leaflet still needs clinical review, legal scrutiny, and source checking before it is used in practice.
For stroke services, that means AI has a role only if humans remain in the approval chain. The safest model is not to replace expert-authored material but to support it. A clinician can catch the missing seizure guidance. A legal reviewer can confirm the language aligns with driving regulations. A patient education team can keep the leaflet readable without stripping out the detail patients actually need.
This comparison leaves one clear conclusion within its own limits: better prose is not the same as safer advice. The strongest AI-generated leaflet came close in readability, but the Stroke Association version still set the standard for factual precision, source quality, and dependable patient guidance on driving after stroke.