Guideline Text Narrowed the Gap Between ChatGPT and Claude
When endodontic care intersects with medication-related osteonecrosis of the jaw (MRONJ), the stakes shift quickly from routine tooth preservation to risk-sensitive decision-making. Patients receiving antiresorptive therapy often present a difficult balance: preserve the tooth when possible, but avoid procedures that could worsen jaw necrosis. This study asked a narrower question with growing practical relevance in dental AI: can a large language model answer more like a guideline-informed clinician when the guideline itself is placed in front of it?
Investigators compared ChatGPT, Claude, and DeepSeek across 18 fictional but standardized MRONJ-related endodontic scenarios built from the 2022 American Association of Oral and Maxillofacial Surgeons Position Paper and current scientific evidence. Two phases shaped the design. In the first, each model answered without added guidance. In the second, the full AAOMS Position Paper was supplied before the same scenarios were repeated. Two endodontic specialists scored anonymized responses across five clinical decision-making domains using a six-point ordinal scale from 0 to 5, and the primary endpoint was the mean of those five domain scores.
A direct pattern emerged at baseline. ChatGPT posted a higher mean overall score than Claude, 4.68 ± 0.14 versus 4.29 ± 0.38. DeepSeek trailed both models throughout the evaluation. The separation between ChatGPT and Claude did not last once guideline-assisted prompting entered the workflow.
Claude Improved Most After Exposure to the AAOMS Position Paper
Claude’s mean overall score rose to 4.68 ± 0.28 after the guideline was provided, a gain of 0.39 points on the 0 to 5 scale. The statistical difference between ChatGPT and Claude disappeared in the second phase. A significant overall phase effect was detected, along with a model-by-phase interaction, showing that the models did not respond to the guideline in the same way.
One sentence captures the clinical consequence: the prompt changed the answer pattern more than the model itself did. In a setting where a guideline functions as an external scaffold, the language model appears less like a standalone decision-maker and more like a rapid retrieval system whose output can be tightened or loosened by the quality of the embedded reference text. For MRONJ, where clinicians must consider tooth retention, infection control, and the danger of provoking further bone injury, that distinction matters in daily practice even if the study intentionally stopped short of claiming real-world diagnostic accuracy.
The largest model-specific improvement appeared in follow-up recommendations, especially for Claude. That domain is clinically meaningful because aftercare in MRONJ-prone patients often determines whether a conservative approach remains safe or whether escalation becomes unavoidable. A model that can better align with post-treatment surveillance advice may be more useful than one that produces polished but generic procedural language.
Baseline Strength Did Not Equal Independent Clinical Validity
DeepSeek consistently received lower overall scores than both ChatGPT and Claude. The study did not present DeepSeek as an outlier of interest for a single domain; instead, it remained the weakest performer across the composite assessment. The domain-level cumulative link mixed model sensitivity analyses generally supported the mixed-model findings, although the phase effect for consultation and medication management was not confirmed in that ordinal framework. In other words, most of the signal held up, but not every domain behaved identically once the data were analyzed through a stricter ordinal lens.
The design choices deserve careful interpretation. The scenarios, reference standard, and guideline-assisted intervention were all aligned with the same AAOMS guideline. That makes the comparison useful for measuring concordance with a predefined standard, but it also means the study does not independently validate clinical correctness in a broader sense. A model can sound more consistent with the guideline it was fed and still fail when confronted with different experts, different institutions, or more ambiguous cases. In dentistry, that limitation is not theoretical; endodontic decision-making often hinges on local risk tolerance, comorbidities, and surgical judgment that are difficult to compress into a fixed prompt.
Follow-up Advice Appeared Most Sensitive to Prompting
The clearest change after guideline-assisted prompting was not a generic uplift across every category. It clustered around follow-up advice, which suggests that structured clinical text may help models when the task demands longitudinal planning rather than a single isolated choice. Follow-up recommendations in MRONJ can include surveillance, reassessment, and caution around additional interventions, all of which depend on a consistent reading of risk rather than improvisation.
This is where the biology and the informatics intersect. MRONJ reflects impaired jaw healing in the setting of antiresorptive therapy, so the clinical logic often favors conservative management, careful medication review, and avoidance of unnecessary trauma. A model that can better echo that logic after exposure to the position paper may assist clinicians by organizing the relevant considerations more reliably, but only as a supervised aid. The study offers no basis for autonomous use at the chairside.
What the Data Support and What They Do Not
The overall finding is straightforward: guideline-assisted prompting improved concordance with a predefined MRONJ decision standard, particularly for Claude, and erased the baseline advantage ChatGPT held over Claude. DeepSeek remained behind the other two models. The phase effect and model-by-phase interaction indicate that prompt context changed performance in measurable ways.
The safety message is equally straightforward. These findings should not be read as proof of clinical accuracy, and they should not be used to justify unsupervised dental decision-making. The scenarios were fictional, the scoring system was built around one guideline, and the same guideline shaped both the prompt and the reference standard. Real clinics do not offer that neat symmetry. Patients bring mixed treatment histories, different levels of oral disease, and competing medical risks that no single position paper can fully normalize.
Further work needs externally developed scenarios and real-world clinical cases before these models can be considered for clinician-supervised decision support. Until then, the most defensible use of large language models in this niche remains constrained: they may help organize guideline-based reasoning, but they cannot substitute for specialist judgment in MRONJ-prone endodontic care.