LLM diabetes remission answers were useful, but never uniformly strong
Large language models are moving quickly into patient education, and type 2 diabetes remission is exactly the kind of subject that exposes both their strengths and their limits. The concept is clinically loaded: patients often hear “remission” and assume cure, while clinicians know the term depends on glycemic thresholds, treatment history, timing, and ongoing surveillance. In that setting, an AI response has to do more than sound polished. It has to be medically correct, reassuring without being misleading, readable without becoming simplistic, and complete without wandering into false certainty.
This cross-sectional balanced factorial study tested exactly that tension. Investigators generated 384 responses to 16 core questions on type 2 diabetes remission using four large language models—DeepSeek-V3.2, GPT-5.5, Gemini 3.1 Pro, and Claude Sonnet 4.6—and six prompting strategies: Keyword, Question, 6th-grade, Guideline, Patient, and Doctor. Two endocrinologists with more than five years of clinical experience each independently scored the outputs across five domains: Accuracy, Relevance, Comprehensiveness, Empathy, and Falsehood avoidance. Readability was then measured with FRES, FKGL, GFOG, and SMOG. Every response was included. No missing data entered the analysis.
One finding stood out immediately: no single model-prompt pairing dominated every endpoint.
Accuracy improved in some models, while empathy rose with different prompts
Across models, the descriptive Composite quality score ranged from 4.175 for Gemini 3.1 Pro to 4.285 for DeepSeek-V3.2. Across prompting strategies, the score ranged from 4.066 for the 6th-grade strategy to 4.350 for the Doctor strategy. Those numbers suggest a narrow overall spread, but the domain-level analysis tells the more important story. Main effects of model were significant across all five quality domains. Accuracy reached LR χ² = 10.964, Relevance LR χ² = 54.042, Comprehensiveness LR χ² = 12.018, Empathy LR χ² = 50.905, and Falsehood avoidance LR χ² = 18.752, with all corresponding P values at or below 0.012. Prompting strategy also exerted significant effects across the same domains, and the largest one appeared in Empathy, where LR χ² reached 243.836.
The magnitude of that prompt effect matters because empathy is not ornamental in diabetes education. Patients asking about remission are often asking about prognosis, reversibility, weight loss, medication tapering, relapse risk, and whether the diagnosis still applies. A technically correct answer that sounds cold can miss the moment entirely. A warmer answer can support engagement, but warmth alone does not prevent misinformation. The study shows that prompt framing strongly influences where an answer lands on that spectrum.
GPT-5.5 had the numerically highest mean scores for Accuracy, at 4.448, and Falsehood avoidance, at 4.469. DeepSeek-V3.2 led Relevance, Comprehensiveness, and Empathy numerically, with mean scores of 4.948, 4.427, and 3.615 respectively. Among prompt styles, Patient produced the highest mean Empathy score, at 4.359, while Doctor produced the highest mean Comprehensiveness score, at 4.609. Those patterns fit clinical intuition: language that mimics a patient-facing exchange can sound more emotionally attuned, while physician-directed framing can push the model toward fuller explanations and broader clinical context.
Readability rose as some quality dimensions moved in the opposite direction
Readability showed its own set of trade-offs. Both model choice and prompt strategy significantly affected FRES, FKGL, GFOG, and SMOG. DeepSeek-V3.2 had the highest mean FRES at 51.28, while Claude Sonnet 4.6 had the lowest at 32.98. Among prompts, 6th-grade produced the highest mean FRES at 69.70, and Doctor produced the lowest at 27.50. In practical terms, the 6th-grade prompt made the language easier to read, but not necessarily more clinically robust. The Doctor prompt tended to generate denser, more technical explanations, which may help with completeness while making the prose harder for lay readers to absorb.
A single sentence captures the central tension: easier prose was not always better medical prose.
In exploratory correlation analyses at the response level, FRES was positively correlated with Empathy, at ρ = 0.53, but negatively correlated with Comprehensiveness, Falsehood avoidance, and the Composite quality score, at ρ = −0.46, −0.40, and −0.26 respectively. Those analyses were explicitly hypothesis-generating and did not account for clustering within Question ID, so they should not be overread. Even so, the direction of the associations is clinically plausible. Shorter, simpler responses can feel more humane and accessible, yet the simplification may strip away nuance about eligibility, thresholds, follow-up, relapse, and the fact that remission is not the same as cure. In diabetes counseling, those are not minor omissions. They are the content that prevents misunderstanding after the patient leaves the visit.
What this means for patient-facing AI in diabetes care
The study’s most consequential message is not that one model won and another lost. It is that patient education on type 2 diabetes remission is structurally resistant to a single “best” AI setting. Accuracy, comprehensiveness, readability, empathy, and falsehood avoidance pulled in different directions, and the winning combination depended on which of those outcomes mattered most. A prompt optimized for plain language could improve readability while weakening completeness. A prompt that pushes for physician-like depth can strengthen coverage but make the response less accessible. A patient-facing style may sound kinder and more engaging, yet it does not automatically produce the most complete or most accurate medical explanation.
For clinicians, that distinction has direct consequences. If AI is used to draft educational content, the content cannot be judged by fluency alone. A polished answer about remission can still blur the difference between remission and cure, or omit the clinical need for ongoing monitoring after glycemic improvement. For patients, the stakes are equally concrete: a readable explanation that leaves out relapse risk or treatment criteria may create false reassurance. For developers, the implication is even sharper. Model selection and prompt design should be treated as separate levers, not interchangeable ones, and neither should be assumed to optimize every dimension at once.
The study is bounded by its own design. It examined a fixed set of 16 questions, relied on expert ratings rather than patient comprehension outcomes, and used an exploratory correlation approach that was not designed to infer causality. Even so, the data show that AI-generated diabetes education cannot be tuned for one metric in isolation. In this domain, the cost of simplicity can be lost precision, and the cost of precision can be harder reading.