Agentic AI in medicine needs guardrails before the damage becomes irreversible
Medicine has spent decades chasing tools that promise to make care safer, faster, and more precise. Agentic artificial intelligence now arrives wrapped in that same promise, but with a different order of risk: systems that can plan, adapt, and act without constant human direction. In this BMJ editorial, the warning is not about distant hypotheticals. It is about a technology powerful enough to affect diagnosis, prescribing, surgery, logistics, and even the environmental footprint of health systems while the rules around it remain behind the curve.
The appeal is obvious. An AI system that can continuously monitor bodies, detect patterns, recommend interventions, and coordinate actions sounds like a clean solution to clinical fragmentation. It could, in theory, reduce prescribing cascades, improve recognition of myocardial infarction in women, speed access to secondary care, and sharpen rare disease data sharing. In practice, the same autonomy that makes agentic AI attractive also makes it difficult to contain. Once a system can revise its own output, choose among options, and pursue goals over time, the possibility of error is no longer limited to a single answer on a screen. It can propagate through an entire workflow.
One line in the piece lands with particular force: the technology may help medicine, but it may also kill us. The phrasing is deliberately sharp because the underlying concern is not abstract. If experts place the odds of catastrophic failure anywhere between 10% and 50%, the uncertainty itself becomes a warning signal. In clinical systems, a tool does not need to be wrong every time to become dangerous. It only needs to be wrong at scale, in the wrong setting, or with enough confidence to pass through human oversight unchallenged.
Agentic AI raises a second problem that hospitals and governments have not yet solved well enough: resource demand. Large-scale development and deployment draw on energy, water, computing infrastructure, and raw materials. That matters to health care because the sector already lives with the consequences of a warming climate, strained supply chains, and environmental exposures that worsen disease burden. An AI system designed to improve care but powered by a chain of resource-intensive infrastructure can erode the very conditions that support health. This is not a side issue. It is part of the clinical equation.
The editorial’s critique widens further when it turns to the practical uses being assigned to AI with almost messianic enthusiasm. Prescribing cascades are already common and harmful. A well-designed system could potentially detect these patterns early, flag inappropriate medication additions, and force a pause before harm accrues. Missed heart attacks in women remain another area where structured intelligence may help by surfacing atypical presentations that clinicians have historically under-recognized. Rare disease genomics could benefit if laboratories shared data more effectively and systems connected fragmented records. Yet none of those gains arrive automatically. A machine that is capable of pattern recognition is not automatically capable of ethical prioritization, contextual judgment, or restraint.
A single sentence captures the central tension: the machine can help, but it can also overreach. In autonomous systems, persistence can become a hazard. If the model keeps pursuing a target after the context has changed, or if it optimizes a metric that does not fully represent patient welfare, then the danger is not merely theoretical bias. It is procedural harm encoded into the workflow. Clinical teams know this logic from other technologies. Decision support that is too blunt becomes alert fatigue. Automation that is too confident creates complacency. And in surgery, the promise of robotic precision has already run into a hard boundary: long-term outcomes have not clearly separated from those of human operators, even as costs have increased.
Regulation has lagged behind capability
The call for governance is not new, but the urgency has changed. The editorial points to existing frameworks and moral codes, from Asimov’s fictional laws of robotics to medical oaths and modern professional pledges. Those documents matter because they reflect a long-standing clinical instinct: power without constraint eventually harms patients. Yet broad ethical statements do not regulate systems on their own. They do not define liability, testing thresholds, audit requirements, model updates, or restrictions on deployment in high-risk environments. Independent oversight has to do that work.
What makes agentic AI different from earlier digital tools is the degree of initiative embedded in the system. Traditional software waits for instructions. Agentic systems may set intermediate goals, choose actions, and adapt their behavior in response to new inputs. In health care, that means a failure can arise not just from a wrong prediction, but from the sequence of actions the machine takes after that prediction. A diagnostic error is serious. An error that triggers treatment, monitoring, escalation, and resource allocation across multiple settings is worse.
The editorial also rejects the reflex to treat speed as virtue. The history of medicine is crowded with technologies that were embraced faster than they were understood. Buxton’s law, quoted in the piece, captures the pattern neatly: it is always too early to evaluate a new technology until, suddenly, it is too late. That logic fits agentic AI well. Hospitals, insurers, regulators, and vendors may be tempted to assume that early enthusiasm can substitute for evidence. It cannot. Evaluation has to keep pace with deployment, not trail it by years.
The most sobering part is the recognition that the harm may arrive in more than one form. Patient safety risk, environmental cost, financial burden, and regulatory capture can all move at once. A technology marketed as medicine’s latest miracle cure can become another source of inequity if wealthier systems adopt it first, patients are exposed before safeguards exist, and commercial incentives outrun public oversight. The editorial leaves no room for passive optimism. It argues for regulation that is independent, urgent, and broad enough to protect human health, nature, and the planet at the same time.
The practical question is no longer whether AI belongs in medicine. It already does. The real test is whether clinicians and regulators can impose boundaries before autonomy, scale, and commercial momentum make those boundaries impossible to enforce.