Large language models produce uneven scoliosis surgery guidance for families

A multi-model assessment of adolescent idiopathic scoliosis education found a clear split between polished prose and practical usefulness. Five large language models answered the same 25 Chinese questions covering the surgical pathway, yet their strengths diverged in ways that matter at the bedside: one model could generate the most reassuring-looking explanations, while another was better at producing instructions families could actually use.

The study focused on a real communication problem in spine surgery. Families preparing for scoliosis correction often want to know what the operation means, what recovery looks like, how to prepare at home, and what warning signs should trigger a call back to the team. When those questions are answered by artificial intelligence, surface fluency alone is not enough. A response can sound confident, read smoothly, and still fail to help a parent prepare for discharge, wound care, pain control, or mobility planning.

What the investigators asked and how they judged the answers

Investigators built 25 standardized questions across six domains through a multidisciplinary working group, a two-round modified Delphi process, and semi-structured parent interviews. Each of the five models accessible in China — GPT-5.6 Terra, Kimi K3, DeepSeek-V4, Qwen3.8-Max-Preview, and Gemini 3.6 Flash — answered every question in Chinese on a single date, 20 July 2026. That produced 125 model-question response units for review.

Two spine surgeons independently scored all responses under blinded conditions using four tools: DISCERN, EQIP, the Global Quality Scale, and the Chinese version of the Patient Education Materials Assessment Tool for printable materials, or C-PEMAT-P. The design choice was important. Because each model answered the exact same questions, the responses could not be treated as independent observations across models, so the results were reported descriptively rather than with inferential testing.

The study was explicit about its limits. It evaluated information quality and patient-education suitability, not factual accuracy, hallucination frequency, or the clinical safety of the recommendations. In an AI setting, that distinction is decisive. A response may score well on readability or structure while still drifting from safe surgical advice, especially when the question involves medication management, recovery timing, or complication recognition.

GPT-5.6 Terra led on quality scores, but not on usability

GPT-5.6 Terra posted the highest mean DISCERN score at 67.92 and the highest Global Quality Scale score at 4.88. Those numbers suggest strong overall informational quality and a polished clinical tone. Yet the same model had the lowest understandability and actionability scores among the five, with C-PEMAT-U at 67.62% and C-PEMAT-A at 30.80%.

The pattern points to a familiar failure mode in language models used for health education. A system can sound authoritative and coherent while still offering advice that is difficult for families to turn into action. Surgical education depends on more than explanation. Parents need clear steps: what to expect before admission, how to prepare for postoperative care, when to seek help, and how to translate instructions into daily decisions. A model that writes elegantly but fails to make those steps concrete leaves a gap between comprehension and behavior.

Kimi K3 was stronger on actionable guidance

Kimi K3 produced the highest EQIP score at 61.05%, along with the top C-PEMAT-U score at 81.44% and the best C-PEMAT-A score at 46.80%. Its outputs were therefore more usable from a patient-education standpoint, even if they did not lead the pack on the broader quality scales.

That split matters because understandability and actionability are not the same thing. A family may read a sentence and understand it perfectly, yet still not know what to do next. Actionability is the more demanding test: can the instruction be carried out in a real home, by real caregivers, under real stress? In scoliosis surgery, where postoperative routines can be complex, that difference has immediate clinical consequences.

Across all five models, the actionability scores were uniformly lower than the understandability scores. C-PEMAT-A means ranged from 30.80% to 46.80%, while C-PEMAT-U means ranged from 67.62% to 81.44%. The gap was not subtle. It suggests that the models were better at explaining than at operationalizing care, which is exactly where families often need the most support.

Quality scores did not move in lockstep with usability

DISCERN and Global Quality Scale scores did not track with the C-PEMAT measures. GPT-5.6 Terra sat at the top of DISCERN and GQS while finishing last on the PEMAT-based usability metrics. Kimi K3 led the usability measures without leading the broadest quality scales.

This disconnect is clinically meaningful. It shows that strong narrative quality does not guarantee patient-centered instruction. In practice, a surgeon or nurse reading an AI draft cannot assume that a high-quality general explanation also contains usable postoperative guidance. The document may need direct editing to add concrete actions, plain-language instructions, and stepwise advice before it can be shared with families.

The finding also argues against a one-model approach. Different systems may be suited to different tasks: one for generating a coherent explanatory draft, another for producing more practical instructions. If clinicians use AI at all in this setting, the output should be treated as a starting point for review, not a finished education package.

Preoperative communication may benefit, but only with clinician review

The surgical pathway for adolescent idiopathic scoliosis is not a single encounter. It spans initial counseling, perioperative preparation, operative expectations, pain management, mobility, discharge planning, and follow-up. Any AI tool used in that pathway has to support all of those steps without drifting into vague reassurance or generic advice.

This analysis suggests a narrow but real role for large language models: they may help structure preoperative communication and prepare families for visits, especially when clinicians want a rapid draft of commonly asked questions and answers. The models, however, should remain adjuncts to specialist counseling. The outputs require review by clinicians who know the patient, the procedure, and the local postoperative pathway.

Two issues remain central before wider adoption. First, factual accuracy still needs direct measurement rather than assumption. Second, patient-centered validation is still missing, including testing with families who must actually use the material at home. In scoliosis surgery, the difference between a fluent answer and an actionable one is not cosmetic; it can shape what happens after the patient leaves the clinic or the hospital.


Reporting by Dr. Aishwarya N., MBBS for Medicine World Foundation, with additional data provided by Frontiers in Immunology.