Canada’s AI push in health care collides with a weak privacy framework
Canada’s federal AI for All strategy arrives with a promise to accelerate artificial intelligence while protecting privacy, confronting algorithmic bias, and supporting Indigenous self-determination over how AI is built and used. The problem is not the ambition. It is the legal scaffold beneath it.
Health and life sciences sit at the center of the government’s AI priorities, and that sector depends on some of the most sensitive data in the country. Clinical records, genetic information, longitudinal care histories, and health service utilization data can all be repurposed to train AI systems, sold to third parties, or embedded in decision-support tools. Once those data move through private-sector pipelines, consent and governance become the difference between legitimate innovation and quiet misuse.
Canada’s proposed Protecting Privacy and Consumer Data Act, introduced as Bill C-36, is meant to replace PIPEDA and govern private-sector data handling. Yet the legislation contains gaps that leave sensitive health data exposed at exactly the moment AI development is accelerating.
Legitimate interest opens a wider door to data processing
Bill C-36 would allow companies to collect, use, and share personal data without knowledge or consent when a company judges it has a “legitimate interest” that outweighs any reasonably foreseeable adverse effect on the individual. In practice, that creates a new basis for processing personal data outside the ordinary consent framework, and it extends beyond the narrower approach used in an earlier reform attempt.
The concern is straightforward: the more permissive the exception, the harder it becomes for individuals to control where their health-related information goes and how it is repurposed. A recent U.S. example shows why that matters. After an AI company acquired Ambry Genetics in 2025, patient data were used to build AI systems and licensed to pharmaceutical companies without patient consent. Canada’s bill includes guardrails, but they do not expressly affirm the human rights of individuals, and the practical meaning of those guardrails will depend on future interpretation.
The European Union’s General Data Protection Regulation draws a firmer line. Legitimate interests cannot override the fundamental rights and freedoms of the individual, and people have a right to object to processing on that basis under Article 21. Bill C-36 contains no comparable right.
Privacy law for AI cannot rely on vague balancing tests when the data are clinical, permanent, and commercially valuable.
Deidentification does not erase the risk of reidentification
Bill C-36 uses a deidentification definition that resembles pseudonymization. Names and direct identifiers may disappear, but indirect identifiers remain. Partial postal codes, rare diagnoses, unusual care pathways, and combinations of attributes can be enough to point back to a person, especially in small or specialized populations.
For Indigenous Peoples, the risks are sharper. The First Nations Information Governance Centre has warned that the bill’s approach can still capture indirect identifiers unique to First Nations communities, including postal codes, remote or fly-in community designations, and claims to programs offered only to Indigenous Peoples. Data can then be aggregated in ways that reveal or infer First Nations status, conflicting with Indigenous Data Sovereignty principles. Bill C-36 is silent on that problem.
A single sentence captures the issue: removing a name does not remove identity from clinical data.
Anonymized data fall outside the law even when they power AI systems
The bill goes further by excluding anonymized data from its scope. Its definition of anonymization requires irreversible and permanent modification so there is no reasonably foreseeable risk of reidentification, directly or indirectly, by any means. Once data are classified that way, the law provides no governance to ensure that reuse is reasonable or appropriate.
That matters because anonymized data are often the raw material for AI development. One example cited in the article involves an Ontario chain of clinics that sells deidentified patient records to IQVIA, an international data broker and AI system creator. Under a similar provincial definition, those records are likely to be treated as anonymized for Bill C-36 as well, which would leave IQVIA’s activities outside regulatory oversight. Deliberate reidentification attempts would still be penalized, but the larger pipeline of use would remain largely exempt.
The policy gap is not abstract. AI systems trained on “safe” data can still generate unsafe outputs.
Bias can survive anonymization because the bias is already embedded in the underlying health system data. The article points to a widely used U.S. algorithm that allocated fewer hospital resources to Black patients than to White patients. When historical patterns of unequal access, diagnosis, referral, or treatment become training data, the resulting model can reproduce and even automate those inequities at scale. The problem is not merely that data can be traced back to a person. It is that data can be misused even when they cannot.
Oversight shifts from an independent commissioner to a new commission
Bill C-36 also removes oversight from the Privacy Commissioner of Canada and transfers it to a new commission that reports to the governing party. That shift matters because privacy enforcement is not only about technical expertise; it is about institutional independence.
The bill aligns with the government’s broader effort to support the AI industry, and its decision-making framework includes not just privacy but also economic and trade treaty considerations. When economic development becomes part of the same apparatus that adjudicates privacy, the risk is that privacy protection is no longer the sole or even primary lens.
Public trust in health AI depends on a regulator that can act without appearing to serve industrial policy first. A less independent oversight body may raise concerns among patients, clinicians, researchers, and Indigenous communities whose data will be touched by these systems.
One concrete proposal is to keep oversight with the Privacy Commissioner of Canada and create a separate tribunal for monetary penalties. That would preserve independent judgment while avoiding conflicts tied to sanctioning the very private-sector actors the commissioner oversees.
AI governance for health data needs tighter consent limits and real Indigenous consultation
The authors argue that exceptions to informed consent should never override the fundamental rights of individuals or Indigenous Peoples. They also call for meaningful consultation with First Nations, Inuit, and Métis communities on the safeguards needed to protect against unconsented use of Indigenous data.
Canada could also look to the European Union for a more complete model. The EU excludes anonymized data from the GDPR, but still limits the purposes for which sensitive data can be anonymized and has separate AI governance legislation for high-risk systems, including those used in health care. That combination of data protection and AI-specific oversight is more robust than a privacy-only approach.
If Canada chooses standards and compliance certification instead of legislation, the work cannot wait. Standards need teeth, and certification needs enforcement. Without them, AI systems built on private-sector health data may continue to expand faster than the rules designed to govern them.
The central question is not whether AI should be used in health care. It already is. The question is whether Canada will build a privacy regime strong enough to keep patients’ data, Indigenous rights, and public confidence intact while that use accelerates.