Online Shockwave Lithotripsy Information Falls Short on Quality and Readability
Shockwave lithotripsy has become a familiar answer for small, uncomplicated urinary stones, but the patient information surrounding the procedure has not kept pace with clinical demand. As kidney stone cases continue to rise, people searching online for plain-language guidance are likely to encounter material that is uneven in quality, inconsistent in sourcing, and often harder to read than a patient leaflet should be.
That gap was the focus of a benchmarking study comparing web-based patient resources with leaflets generated by artificial intelligence. The exercise was straightforward in design and clinically pointed in purpose: measure whether information about shockwave lithotripsy meets standards that matter to patients and carers at the moment they are deciding whether to proceed with treatment.
What the Benchmarking Study Measured
Investigators collected the top 20 search results for four SWL-related terms across Google, Yahoo, and Bing, then removed duplicates. From that search pool, 68 websites were included. Generative AI platforms, including ChatGPT, Gemini, and DeepSeek, were also used to create a patient leaflet. Two authors then assessed the material using established instruments for quality and readability: DISCERN, JAMA benchmark criteria, the Flesch Reading Ease Score, and the Flesch-Kincaid Grade Level.
Those tools matter because they capture different parts of the same problem. DISCERN examines the reliability and quality of treatment information. JAMA benchmark criteria ask whether a site makes its authorship, attribution, currency, and ownership clear. Flesch Reading Ease and Flesch-Kincaid Grade Level estimate how difficult the text is to read, which is especially important when patients are trying to understand a procedure that sounds technical even before the details are explained.
A second pass added another layer: ChatGPT reviewed the AI-generated information using DISCERN, creating a comparison between human-rated and AI-rated quality. In a field where patients may increasingly encounter machine-generated explanations before they ever reach a clinician, that comparison is not academic. It is a test of whether automated tools can reliably police their own output.
Quality Scores Were Moderate, But Transparency Was Thin
The overall quality picture was mixed. Average websites and AI leaflets produced DISCERN scores of 52.4 ± 8.91 and 58.7 ± 5.03, respectively. JAMA benchmark scores were 2.22 ± 1.10 for websites and 0 for the AI leaflets. Readability was similarly close across both groups, with Flesch Reading Ease scores of 52.1 ± 13.8 and 51.5 ± 8.05, and Flesch-Kincaid Grade Levels of 9.04 ± 2.36 and 8.93 ± 1.10.
The most important signal came from the transparency measure. JAMA benchmark differences were statistically significant, with P = 0.0010. The other comparisons were not significant: DISCERN P = 0.224, FRES P = 0.652, and FKGL P = 0.866.
A score near ninth-grade reading level is not catastrophic, but it is still above the reading threshold many patient education experts aim for when they want material to be genuinely accessible. For a procedure discussion, that extra complexity can change whether patients understand the trade-offs, the possible need for repeat treatment, or the limits of symptom relief.
The zero JAMA benchmark score for the AI leaflets is the sharpest warning in the dataset. Even when generated text is readable and reasonably organized, it can still fail on basic standards of disclosure and trustworthiness. In practice, a patient may see fluent prose but still have no clear sense of who produced it, what evidence supports it, or whether the content has been updated.
Why the Gap Matters at the Bedside
SWL is often presented as a noninvasive option, but noninvasive does not mean simple. Patients still need to understand stone size, stone location, expected success rates, possible need for retreatment, and the chance that symptoms may persist if fragments do not clear. Clear patient information supports informed consent, reduces uncertainty, and helps people prepare for recovery and follow-up.
When online material falls short, the burden shifts back to the clinic. That can slow decision-making, amplify anxiety, and leave patients relying on fragments of information pulled from search results rather than a coherent explanation anchored in clinical reality.
The study also points to a deeper issue in the age of generative AI. These systems are trained on large amounts of online text, so they can reproduce the strengths and weaknesses of the material they learn from. If the underlying web content is thin on citations, authorship, and update dates, machine-generated leaflets may inherit those defects unless they are actively corrected.
Human reviewers may still outperform machines on one of the most important questions in patient education: can this information be trusted? The discrepancy between author-rated and AI-rated DISCERN scores suggests that automated self-review is not yet a substitute for critical human oversight.
Readability Alone Is Not Enough
The data make one point especially clear: readable text and reliable text are not the same thing. A patient leaflet can sit at roughly the right reading level and still fail to identify its source, describe evidence transparently, or disclose authorship and review status. Those omissions are not cosmetic. They are part of how trust is built in patient-facing medicine.
For clinicians, the practical implication is immediate. Online patient resources about shockwave lithotripsy should be checked not only for readability, but also for provenance, currency, and evidence quality. For developers of AI-generated educational content, the standard has to be higher than grammatical fluency. Patients need material that is both understandable and verifiable.
The search-based design of the study also reflects the way patients actually look for help. Most do not begin with a journal abstract or a specialist society leaflet. They start with a search engine and move through whatever appears first. When the first layer of information is inconsistent, the entire decision pathway becomes less secure.
What This Means for Patient Education Resources
The findings support a straightforward clinical conclusion: online patient education for shockwave lithotripsy remains uneven, and generative AI does not automatically solve that problem. The central deficiency is not just complexity; it is quality assurance. The most useful educational tools will need evidence-based content, plain language, transparent attribution, and a review process that does not depend on the same model that produced the text in the first place.
The study also leaves a useful caution for health systems and professional societies that are considering AI-assisted patient materials. A polished leaflet can still be incomplete, and a moderate readability score can still conceal weak sourcing. For a procedure used in a growing stone disease burden, those shortcomings matter at the point where patients are making real choices about treatment.
The unanswered issue is whether future AI-generated resources can be built with better guardrails than the public web that currently feeds them. Until then, the quality gap remains visible in the scores, and the transparency deficit remains the most troubling finding in the comparison.