AI chatbots outscored clinicians on radiation safety knowledge in a 25-item test
Radiation safety knowledge remains uneven across the people most likely to counsel patients about exposure, and a new comparison study found a striking gap between human respondents and large language model chatbots. Using a 25-item multiple-choice questionnaire on radiation risks and radiation safety, investigators tested 344 people drawn from dentists, medical doctors, and the general public, then entered the same questions into seven AI-based chatbot systems: Claude, DeepSeek, Microsoft Copilot, Gemini, Perplexity AI, ChatGPT-5, and ChatGPT-4o. The result was blunt. Across the evaluation conditions used here, the chatbots answered more questions correctly than the human participants.
The performance spread was wide. AI-based systems achieved accuracy rates between 80% and 90.67%, while human participants landed between 32% and 60%. Microsoft Copilot produced the strongest overall accuracy, and Perplexity AI had the weakest showing among the chatbot group. Those numbers do not describe bedside competence, diagnostic nuance, or the ability to weigh competing risks in an individual patient. They do, however, show that on a fixed knowledge test about radiation safety, the machines were more consistent and more accurate than the people surveyed.
Dentists and medical doctors did better than the public, but both groups still left room for error
Among the human participants, dentists and medical doctors performed significantly better than members of the general public, yet no significant difference emerged between the two healthcare professional groups. In practical terms, the professional training expected to include radiation-related counseling did improve performance, but not enough to close the distance to the AI systems. That gap matters because radiation questions arise in everyday care, especially around imaging referrals, informed consent, cumulative exposure, and patient reassurance. A clinician who misunderstands a basic safety principle can misinform a patient at the exact moment when clear guidance is most needed.
The result also points to a familiar problem in medical education: radiation safety is often spread across dental training, medical school, imaging departments, and continuing education, rather than reinforced as a tightly integrated core skill. When knowledge is fragmented, performance tends to drift. A multiple-choice format may seem simple, but it is often enough to expose whether foundational concepts have truly stuck. Here, the human scores suggest that the baseline remains inconsistent.
One sentence captures the core finding: the AI systems answered more radiation safety questions correctly than dentists, doctors, or the public.
Stable chatbot performance raises one question, while one model broke the pattern
Performance remained stable across morning, midday, and evening assessments for most of the AI systems, suggesting that the evaluated chatbots did not meaningfully lose accuracy over the time windows tested. ChatGPT-5 was the exception, showing a significant temporal difference with _p_ < 0.001. The article does not frame that fluctuation as a clinical warning sign, but it does show that even highly capable models may not behave identically across repeated use. For educational settings, consistency can matter almost as much as peak accuracy, especially if users rely on a tool for repeated reinforcement of the same safety concept.
The comparison was straightforward rather than elaborate: the same questions were sent to human respondents and to each chatbot system, and the answers were scored for accuracy. That design makes the headline result easy to interpret, but it also keeps the experiment tethered to a narrow task. Real-world radiology communication is not a static quiz. It involves uncertainty, patient anxiety, prior exposures, pregnancy considerations, modality-specific risks, and clinical tradeoffs. A model that performs well on a defined question set may still falter when the conversation becomes contextual, emotionally charged, or incomplete.
Still, the pattern is hard to dismiss. AI systems are now accurate enough on some knowledge tasks to outperform trained professionals, at least under test conditions that reward recall and recognition. In radiation safety education, that could make them useful as practice tools, rapid-reference aids, or supplemental teaching resources. The biological rationale is simple in a different sense: radiation harms are tied to dose, timing, tissue susceptibility, and cumulative exposure, so any tool that helps clinicians remember those principles more reliably may improve the quality of counseling before imaging or treatment decisions are made.
Educational value is real, but accuracy on a quiz is not clinical judgment
The strongest caution in this study is embedded in its own conclusion. Better quiz performance does not mean better clinical judgment. A chatbot can retrieve an answer quickly and still fail to understand context, weigh contraindications, or recognize when human escalation is required. Radiation safety in practice is not only about knowing the right definition or threshold; it is about applying that knowledge to a living patient whose risks and benefits change from one encounter to the next.
The study nevertheless has clear implications for teaching. If dentists and medical doctors scored below the AI systems on a targeted knowledge test, then radiation safety curricula may need more repetition, better reinforcement, or more frequent refreshers after formal training ends. The general public’s lower performance is less surprising, but it reinforces the need for clearer patient-facing communication in clinics, imaging departments, and dental offices. Patients often hear the word “radiation” without enough explanation of magnitude, purpose, or risk minimization, and that gap can shape decisions as much as the scan itself.
Ethics approval for the work came from the Mersin University Institutional Review Board and Ethics Committee, and the authors declared no competing interests. The study was conducted in Turkey by a team from the Faculty of Dentistry at the University of Mersin. For now, the main unanswered issue is not whether AI can beat humans on a structured radiation safety quiz, because it did. The more important question is how far that knowledge can be trusted once the encounter stops looking like a test and starts looking like a patient conversation.