AI Confabulation in Consequential Canadian Contexts

AI systems are being deployed as authoritative information sources across Canadian institutions and used by millions of Canadians — in tax administration, consumer services, legal proceedings, and health information — without accuracy verification before deployment and without monitoring after.

The Canada Revenue Agency spent $18 million on a chatbot ("Charlie") that processed 18 million taxpayer queries. The Auditor General found it answered only 2 of 6 test questions correctly. Air Canada deployed a customer service chatbot that fabricated a bereavement fare discount policy; the BC Civil Resolution Tribunal held Air Canada liable for its chatbot's representations. In Quebec, a court imposed the first judicial sanction for AI-hallucinated legal citations when a self-represented litigant submitted fabricated case law generated by a generative AI tool.

The Canadian Medical Association's 2026 Health and Media Tracking Survey (conducted by Abacus Data with 5,000 Canadians in November 2025) documents that 52% of Canadians use AI search results for health information and 48% use them for treatment advice. Those who follow AI health advice are five times more likely to experience harms: confusion about health management (33%), mental stress or increased anxiety (31%), delay in seeking medical care (28%), lower trust in health professionals (27%), difficulty discussing health issues with healthcare providers (24%), strained personal relationships (23%), and avoidance of effective treatments due to misinformation (23%). Despite these outcomes, only 27% trust AI for health information — meaning a large proportion use tools they do not trust, likely driven by access barriers to professional health advice.

The consistent pattern: an institution, platform, or individual deploys AI as an authoritative source, treats its outputs as reliable, and discovers only after harm that the system confabulates. This pattern scales with deployment — as more institutions and individuals adopt AI information systems, the frequency of consequential confabulation increases proportionally.

Some institutions have taken corrective action following documented incidents. The CRA updated its chatbot after the Auditor General's report. Air Canada revised its customer service AI policies after the tribunal ruling. Several AI developers have implemented accuracy improvements and added citations to their outputs. The trajectory of these responses suggests institutional learning, though the pace of correction varies significantly across sectors.