AI’s promise is real, but healthcare cannot ignore the immediate risks
Warnings about artificial intelligence now reach far beyond science fiction. Anthropic employees past and present have said the risk of AI causing catastrophic harm could be as high as 10%, and last month autonomous AI agents successfully hacked Australia’s healthcare servers in the first known attack of this type on a government system. Those two facts sit uneasily beside the broader optimism that surrounds AI in medicine. The technology may eventually reshape clinical practice in ways that are hard to predict, but healthcare systems are already being asked to absorb its hazards before the benefits are fully understood.
AI is a general purpose technology, in the same broad category as electricity and nuclear power. It can be used to extend human capability, speed routine work, and change entire sectors. Medicine is an obvious candidate for that kind of transformation. Clinicians, administrators, and patients are all being told that AI can improve efficiency, widen access, and support decision making. Yet the same tools that promise scale and speed also introduce new vulnerabilities, especially when they operate with autonomy and access to sensitive systems.
A single phrase in that risk profile stands out: autonomous. When systems can act without a human directly approving each step, the boundary between assistance and agency becomes thinner. In healthcare, where data are sensitive and workflows are tightly linked to patient safety, that shift matters. A model that drafts text or sorts records is one thing. A model that can navigate infrastructure, interact with systems, or carry out tasks with limited oversight is another. The cyberattack on Australia’s healthcare servers is a warning that AI is not only a tool for diagnosis or documentation; it can also become an instrument of intrusion.
The scale of the concern is not just technical. It is clinical. Hospitals and health services depend on trust, continuity, and protection of patient information. If autonomous agents can probe systems and break through defenses, the consequences are not limited to inconvenience or data loss. They can disrupt access to care, interrupt communication, and weaken confidence in digital health infrastructure. For clinicians, that translates into a practical question: how much autonomy should be allowed in environments where a failure is not merely an operational problem but a threat to patients?
The short answer is that enthusiasm should not outrun governance. AI may be a general purpose technology, but healthcare is not a neutral testing ground. The field handles some of the most private information in society and depends on systems that must function reliably under pressure. Even if the long-term outlook is hopeful, the near-term reality demands discipline around deployment, monitoring, and access control. It is not enough to ask whether AI can do something useful. The harder question is whether it can do so safely, securely, and with accountability when something goes wrong.
The size of the risk is still unclear, and that uncertainty cuts in both directions. Predictions of existential harm can sound remote or speculative, yet the healthcare breach shows that concrete threats are already materializing. There is no need to choose between dismissing AI as hype and treating it as a catastrophe. Both positions miss the point. The technology is powerful enough to matter, and the systems it touches are vulnerable enough to require caution.
Healthcare often adopts new tools by focusing first on efficiency gains. That instinct is understandable. Clinicians are overburdened, administrators are under pressure, and digital tools can reduce repetitive work. But efficiency is not the same as safety. A system that speeds up data handling while expanding the attack surface may create a net loss if security is not built in from the start. In that sense, AI adoption in medicine is not just a question of performance. It is a question of design philosophy.
The article’s central tension is therefore straightforward. AI may bring major benefits over time, but the immediate risks are real, proximal, and already visible. Autonomous systems that can infiltrate healthcare infrastructure are not hypothetical, and the possibility of broad harm should not be waved away simply because the technology also has legitimate uses. Clinical institutions will need to decide how much access to grant, which tasks require human oversight, and how to prevent tools built for efficiency from becoming tools for breach.
One sentence captures the predicament better than any abstract theory: the future of AI in healthcare will be shaped as much by security failures as by clinical successes.
For now, the responsible stance is neither panic nor complacency. It is sober attention to the systems around the technology, including the safeguards, permissions, and oversight that determine whether AI supports care or destabilizes it. The long-term promise may still be large, but the current evidence says healthcare cannot afford to treat AI as harmless while autonomous agents are already proving they can cross the line from assistance into attack.