Part of the Canadian AI Sovereignty Series
She found out after the eviction order was already signed.
A woman in Brantford, Ontario lost her home because benefits the Ontario Disability Support Program was supposed to pay never arrived. By the time she discovered the payments had failed, the Landlord and Tenant Board had already heard her case and ordered her out. The money had stopped, and no one told her. The reasons stayed hidden. Throughout the process that cost her the roof over her head, she had no way to learn whether a human being or an automated system was responsible for the failure.
This is a documented outcome from the mutual aid work underpinning this series, and it is the shape of a risk that Canada’s new national AI strategy leaves untouched.
AI for All and the Regulation Gap
Prime Minister Carney launched “AI for All” on June 4, 2026. The strategy makes the federal government an explicit strategic anchor customer and commits to accelerating AI procurement and delivery across departments through a new Office of Digital Transformation. It takes what its own observers describe as a measured approach to regulation, with no comprehensive AI-specific framework attached. Speed is funded, scrutiny is deferred.
For more than a year, the recurring worry across this series was procurement moving ahead of strategy. The strategy has now arrived, and the gap it leaves open is wider than before. Three problems sit at its center: procurement designed for speed rather than scrutiny, privacy protections that have fallen behind the technology, and the combination of datasets across jurisdictions and borders into profiles that no single program, intentionally or not, set out to create.
Canada operates well short of a surveillance state. Yet people on benefits face some of the most onerous surveillance and clawback provisions in the country, if not the world. Wealthy people have many ways to shelter income. Benefits are easily traced and easily clawed back, with devastating effect on the people who can least afford it. Disability policy financially punishes disabled people in relationships and can force dependence on a partner, building a high-risk power imbalance directly into its design. The argument here is that the data already collected, combined with the triangulation and distillation tools available today, can produce surveillance without anyone ever deciding to build it.
The risk has five conponents:
- Benefits systems are being automated.
- Recipients are not consistently told when automation affects them.
- Benefits data is becoming more interoperable.
- Cross-border legal channels are expanding.
- Freedom-of-information rights are under strain.
Each part is defensible on its own. Combined, they produce the danger. What follows takes each in turn and applies the concern where the consequences run most severe and least visible: the administration of income support. People on benefits sit among the most exposed to automated error in government, and they hold the fewest resources to contest it. When a payment stops, the question of whether a human or a model made that decision has little available recourse, and can determine whether a family makes rent and stays housed.
Background
Federal automation across benefits delivery is already an announced direction of travel. Employment and Social Development Canada’s 2026-2027 Departmental Plan commits to a new Digital Roadmap that leverages artificial intelligence to automate internal processes and streamline operations, alongside a planned reduction of roughly 1,500 full-time equivalents between 2026-27 and 2028-29. The same plan describes automating program processes within the Canada Pension Plan Disability stream and modernizing Service Canada delivery. Old Age Security and CPP-Disability have migrated onto the Cúram case management platform, and Service Canada has framed increased automation as a response to workload pressure that experienced medical adjudicators can no longer absorb at volume.
The Canada Disability Benefit, administered by Service Canada with first eligibility in June 2025 and payments beginning July 2025, depends on the Disability Tax Credit as its gateway and on assessed tax data for eligibility. It reaches working-age people with severe and prolonged impairments, the precise population for whom a wrongful stoppage is hardest to survive and hardest to appeal.
The Ontario Disability Support Program operates at the provincial level with far less public visibility into its systems than the federal programs carry. This series has documented direct cases. They are not hypotheticals drawn from policy papers. They come from the mutual aid work underpinning this research, and read in sequence they describe a process that moves in one direction only: toward the recipient, and faster than the recipient can respond.
The Brantford case opens the pattern, a woman who lost her home after ODSP payments failed and who learned of the failure only once the Landlord and Tenant Board had already ordered her eviction.
A second case follows the same arc, with a different ending only because strangers intervened. After her payments stopped reaching her assisted-living residence, a woman received eviction documentation, and the outstanding amount was cleared through donations, which is to say that charity resolved what the system responsible for the failure could not.
A third case is unfolding as this goes to publication. In Hamilton, a family’s payments remain unconfirmed, eviction paperwork has been filed, and a Landlord and Tenant Board hearing is scheduled for August, this after seventeen years in the home with ODSP making the payments directly throughout. A system that paid reliably for seventeen years has stopped confirming that it is paying at all, and the clock is now running against the family in a tribunal that will not wait for the confirmation to arrive.
The damage reaches past stopped payments and into outright denial. Two further recipients have had benefits and outstanding amounts from CPP and the Canada Disability Benefit refused on grounds that appear arbitrary, the refusal compounded by the near impossibility of appeal when reaching a human being is close to impossible and the automated processes are themselves broken.
Taken together, the cases establish something narrower and more damning than proof of cause: when payment systems fail, recipients have no practical way to identify the source before the housing consequences begin. The same sequence repeats in each one. A recipient learns of the failure only after the harm is irreversible or nearly so, the eviction process outruns any ability to discover and correct the error, and throughout it all the recipient is told nothing about whether a human or an automated process produced the stoppage, the denial, or the silence.
One absence connects the federal expansion to these provincial cases, and it is disclosure. Canada publishes nothing, in any consistent or auditable form, on the extent to which automated decision systems determine, suspend, or terminate individual benefit payments across federal, provincial, and municipal programs, or by the outsourced vendors those governments contract. The expansion of automation is on the record. The place where a model sits in the decision chain for any given stoppage stays off it.
A Structural Sovereignty Risk
Three forces converge into a structural sovereignty risk: expanding undisclosed automation in benefits administration, the absence of an AI-specific regulatory framework in the new national strategy, and pending legislation that would widen cross-border data sharing while narrowing freedom of information. That risk falls hardest on benefits recipients, and it carries the capacity to convert routine administrative data into an instrument of surveillance without any single decision-maker ever choosing to build a surveillance system.
Evidence
Automation without disclosure produces uncorrectable error
AI systems make mistakes as a structural property, not as an occasional defect. They operate on probability rather than fact, which separates them from the deterministic automated systems Canadians are accustomed to, though those older systems remain capable of enormous harm when poorly deployed, as the Phoenix payroll disaster demonstrated. The probabilistic nature of these tools makes their error predictable in the aggregate even when any single output cannot be anticipated. Evans’ Law, referenced throughout this series and developed through empirical research, formalizes one face of that predictability: the longer a model reasons, the greater the likelihood that its response will be incorrect, until the point at which an incorrect answer becomes more likely than a correct one. The law describes the predictability of error rather than the reliability of the system, and it is the predictability that matters in a benefits context, because a system whose errors are structural rather than incidental needs a structural place where a human catches them. When a model contributes to a benefits decision without human oversight positioned to catch and reverse its errors, those mistakes carry consequences that resist undoing on the timelines that matter to a recipient.
An incorrectly flagged eligibility review, a misread income figure, or a wrongful suspension can take weeks or months to correct through appeal. Rent does not wait for an appeal. Eviction proceedings move faster than administrative review, and the documented cases trace exactly that gap between the speed of a payment failure and the speed of human remedy. Whether automation caused any individual stoppage is precisely what the recipients cannot find out, which is the disclosure problem in its sharpest form.
Accountability begins with disclosure. A recipient told only that their payment has stopped, a payment that they and often their family depend on to stay housed, fed, and medicated, cannot meaningfully contest a decision whose origin is hidden from them. They have no way to know whether to challenge a human judgment, a data error, or a model output. The absence of disclosure is itself a denial of the ability to seek correction.
A pointed asymmetry runs through the housing cases described above. The Landlord and Tenant Board, which decides whether a tenant is evicted, operates under a Tribunals Ontario practice direction stating that adjudication is a human responsibility, that members do not use AI to write decisions or analyze evidence, and that they remain fully accountable for their decision-making. The related Ontario Land Tribunal went further in March 2026, requiring parties to disclose any use of AI to generate content in documents filed with it. The legal forum that hears the eviction now carries a human-decision standard and an AI-disclosure rule. The income-support system that determines whether the tenant could pay rent in the first place carries neither. Governance has been written for the downstream hearing and withheld from the upstream payment decision that drives people into it.
Canada’s immigration system already shows the disclosure failure at scale
The pattern described here for benefits is visible today in Canadian immigration processing, where it has run further and generated a clearer record. Immigration, Refugees and Citizenship Canada introduced an automated processing tool called Chinook, which extracts applicant information into a spreadsheet format and offers officers standardized language to approve or refuse multiple cases at once. Chinook surfaced through Access to Information requests rather than proactive disclosure, with no prior public consultation, a demonstration of the disclosure problem at the heart of this series.
The consequences are now arriving in court. The Federal Court quashed a reported 2,000-plus refusals in 2025 as unreasonable or procedurally unfair, forcing IRCC to reprocess those files, and immigration filings have come to dominate the court’s docket, reaching 24,667 in 2024 from 6,424 in 2020. Immigration lawyers report refusals that fail to engage with the documents on file, including a case where an application was refused for a missing birth certificate that was in fact attached. Other reports describe refusal reasons generated at the same timestamp as the application was processed, which raises direct questions about whether meaningful human review occurred at all. IRCC’s first AI Strategy, published in February 2026, affirms that no tool can refuse or recommend refusing an application on its own. That statement may describe how the system is designed. Whether it describes how the system behaves at volume is the question that keeps arriving in Federal Court.
The structural lesson transfers directly to benefits. A standardized tool that lets an officer move quickly through many files, combined with templated reasons and no disclosure of the tool’s role in any individual decision, produces exactly the conditions seen in the ODSP and CPP cases: a decision the affected person cannot trace, cannot understand, and cannot effectively appeal. Immigration adds a further danger this series has tracked throughout, because immigration is the domain where automated profiling and cross-border enforcement most directly converge.
The American template shows where undisclosed automation leads
The United States offers a working model of what consolidated, automatically targeted benefits-adjacent data becomes once it operates without transparency or independent audit. The instructive feature is the cascade. A flag raised in one system triggers action across agencies, with few mechanisms for correction, no public information on error rates, no independent audit, and minimal congressional oversight.
Palantir’s ImmigrationOS, built under a contract with US Immigration and Customs Enforcement that has grown past $145 million within the existing Investigative Case Management deal, pulls together data from the IRS, the Social Security Administration, passport records, and license plate readers to identify and prioritize individuals for enforcement. A related tool reported as ELITE ingests Medicaid data to generate dossiers and leads on people the agency believes may be deportable. Palantir now holds contracts with both the enforcement arm and the benefits arm of US immigration, which places a single vendor on both sides of the system. Benefits data, gathered for the administration of support, becomes an input to enforcement.
This is the architecture Canada risks importing, and the boundary is what matters now. Canada already shares sensitive person-level immigration, biometric, biographic, and border-movement data with US authorities, while simultaneously moving domestic benefits administration toward automation, fraud analytics, and broader internal data-sharing. These remain distinct systems, which is exactly why the line between them deserves protection. The concern stands without any proof that ODSP, OW, CPP-D, or Canada Disability Benefit payment records already cross the border. A benefits recipient should not have to trust that economic status, disability status, immigration status, fraud suspicion, and identity data will remain separated while the architecture of government is being redesigned around data mobility.
Pending legislation widens the cross-border channel
The relevant Canadian legislation is Bill C-22, the Lawful Access Act, which received first reading on March 12, 2026. Its provisions originated in Bill C-2, the Strong Borders Act, and were split out in late 2025 after the lawful-access measures drew sustained criticism, with immigration provisions retained in Bill C-12. Bill C-22 would compel telecommunications and other electronic service providers to confirm without a warrant whether an individual uses their services, require providers to retain metadata for a year, and expand information sharing with foreign governments, including the United States.
Bill C-22 is advancing while the Canadian government sits in closed-door negotiations with the United States over a bilateral law enforcement data-sharing agreement under the US CLOUD Act. The CLOUD Act is the binding constraint this series has identified repeatedly. Data residency in Canada does not cure it, because the Act reaches data held by US-linked providers regardless of where that data physically sits. The federal government has offered Parliament and the public no explanation for why the United States has pressed Canada to pass these surveillance reforms, and in May 2026, leaders of US Congressional committees wrote directly to Canada’s Minister of Public Safety about the bill.
The conditions for Canadian administrative data to reach US law enforcement assemble in sequence. Benefits data, or the broader administrative data of recipients, moves through non-sovereign infrastructure subject to the CLOUD Act; Bill C-22 widens the lawful channel for cross-border sharing; and cross-referencing against systems like ImmigrationOS becomes possible. People who are in the United States legally, people who are in Canada without status, and people with no enforcement exposure at all become subject to a profiling apparatus that no Canadian program consented to feed. Critics argue that Bill C-22 would expand access to subscriber-identifying information and metadata through new lawful-access mechanisms, while requiring service providers to maintain the capabilities that make such access easier to execute.
Combination is the surveillance mechanism
Data combination is itself the surveillance mechanism, and it operates without any need for a surveillance state to exist. The triangulation and distillation tools available today can produce one from inputs that are each individually mundane. Benefits administration data. AI prompt and search query data. Search engine request data. Advertising serving and response data. Social media data. Layered together, these permit a triangulation that defeats privacy protections built for each dataset in isolation.
A profile assembled from a person’s income support history, their search behavior, their queries to AI systems, the ads served to them, and their social media activity reveals far more than any single source. The combination is the insidious form, and it requires no malicious actor. It requires only that the datasets become combinable, that the legal barriers to combining them fall, and that no framework prohibits the combination. Bill C-22, layered onto a CLOUD Act agreement, layered onto undisclosed automation running on non-sovereign infrastructure, supplies exactly those conditions. Canada holds no legislation specifically governing this form of cross-referenced surveillance.
Visibility decreases as decisions descend through the layers of government
Accountability requires visibility, and visibility is distributed unevenly. Federal programs offer the most insight, through departmental plans, the federal AI use case inventory, and the Office of the Information Commissioner. Provincial visibility runs limited. Municipal visibility approaches absent. The Ontario provincewide medical records announcement of March 2026 illustrates the provincial gap, a major commitment with no timeline, no dedicated funding, and voluntary adoption, leaving a live provincial sovereignty gap with little public oversight of the systems meant to carry it.
This gradient matters because benefits administration spans all three levels. ODSP is provincial. Municipal social services administer their own programs. The federal CDB, CPP-Disability, and Old Age Security sit at the most visible level, and even there the disclosure of automation in individual decisions remains inadequate. As decisions move toward the levels where citizens hold the least insight, the probability of undisclosed and uncontestable automated error rises with them.
Freedom of information is contracting at the moment it is most needed
The federal government is running concurrent reviews of the Access to Information Act and the Privacy Act. The Access to Information review, with submissions due June 15, 2026, examines a regime that already ranks poorly against international standards and that the Information Commissioner has repeatedly had to enforce through Federal Court. The Privacy Act review proposes, among other changes, replacing the definition of personal information with a broader concept of personal data that would remove the current requirement that information be recorded in order to be protected, a change with complex implications for the right of access.
The concern is directional. At the precise moment when citizens need stronger tools to discover how automated systems decide their benefits, the access regime sits under review with proposals that could narrow disclosure. A reduction in freedom of information at the federal level, set against the existing opacity at the provincial and municipal levels, would leave recipients with even fewer means to learn why a payment stopped.
Weighing the Argument Against
Several considerations weigh against the strongest form of this argument, and they deserve a fair hearing.
The federal government maintains an AI use case inventory and operates under the Directive on Automated Decision-Making, which requires algorithmic impact assessments for federal automated decision systems. These instruments deliver more transparency than exists in many jurisdictions, and they impose real constraints on federal practice. The Chinook litigation shows that judicial review remains an available and partly effective check, since the Federal Court has in fact quashed thousands of flawed refusals. At the federal level, the disclosure gap is narrower than this paper’s general framing might suggest, and the courts retain the power to correct individual decisions after the fact.
The reach of that reassurance stops where the documented harm begins. The Directive binds federal automated decision systems. ODSP is provincial. Municipal social services fall outside it entirely. Every case described above occurred at exactly the layer the federal safeguard does not touch, which is why federal transparency, real as it is, offers the Brantford and Hamilton families nothing.
Automation also carries no inherent harm, and the workload pressures driving it are genuine. CPP-Disability has missed service standards and inventory reduction targets, and applicants wait too long for adjudication by overextended medical staff. Well-designed automation with human oversight positioned at the decision points that matter could lower wait times and error rates rather than raise them. The problem identified here is the absence of disclosure and oversight, not the presence of automation as such.
Bill C-22 has not passed, which tempers the cross-border scenario. Its predecessor failed to reach committee under public pressure, and the bilateral CLOUD Act agreement remains unconcluded. The surveillance scenario described here stays conditional on legislative and diplomatic outcomes that are contested and reversible. The combination scenario likewise requires multiple datasets held by different parties under different legal regimes to become combinable, and existing privacy law, imperfect as it is, still imposes barriers to several of the specific data flows the triangulation argument depends on. The apparatus is not assembled, and assembling it would require multiple barriers to fall in sequence.
These points temper the timeline and the certainty. They leave the structural concern standing. A risk that is conditional and reversible is still a risk worth naming before its conditions are met, because the strategy that should address it has chosen to defer regulation rather than build it.
What Has to Change
The benefits layer is where the cost of deferring regulation will be paid first and felt most. “AI for All” funds the acceleration of public sector AI procurement while leaving the regulatory framework, the privacy protections, and the rules on data combination for later. In the administration of income support, later arrives too late for the recipient whose payment has already stopped. Three requirements follow, and they are the minimum a system that can stop a person’s income owes the person it stops.
A human being has to be reachable, quickly and easily, and fastest of all when the matter is urgent. As more systems automate, the ability to reach a person is steadily disappearing, and many systems now offer no way to speak to anyone at all. A guaranteed channel to a human must be mandatory in any system where AI is responsible for critical human outcomes, and arguably in basic customer service as well. When the decision controls rent, medication, or food, the channel has to open in hours, not weeks. This recourse mechanism is almost entirely absent from AI regulation anywhere in the world, and it is among the most critical safeguards there is.
Every decision has to carry a real appeal, and decisions made at speed have to be reversible at speed. A system that can suspend or deny a benefit must pair that power with a mechanism to challenge it and have a human intervene, correct the error, and reinstate a life-critical payment. The faster a system can act against a recipient, the faster its errors have to be correctable, because automation cannot serve as an excuse to sever execution errors from their fixes and leave them standing while an eviction clock runs.
Changes to legislation, practices, and staffing have to be communicated to the people they affect. When a program alters how decisions are made, reduces the human resources behind them, or shifts onto new systems, as ODSP recipients have experienced without explanation, the people receiving benefits are entitled to understand what has changed behind the scenes. A person should always be able to ask and to know, quickly, what information about them sits in which systems, how it is used, where it is shared, and what mechanism produced the decision affecting them. Disclosure of this kind is the precondition for every other protection, because a recipient cannot contest a process whose existence was never disclosed.
The surveillance state is not being chosen. It is being assembled, one defensible decision at a time, by people who would each deny they were building it. A national AI strategy that addresses procurement speed while leaving disclosure, privacy, and data combination unresolved has identified the wrong priority. The people on the benefits layer will be the first to learn what that choice costs.