Auditor General Found CRA's $18-Million AI Chatbot Gave Incorrect Tax Answers

The Canada Revenue Agency launched Charlie, an AI-powered chatbot, in February 2020 to answer taxpayer questions about tax filing, benefits, and CRA services. By the time of the Auditor General’s review in December 2025, the system had processed over 18 million questions at a cost of approximately $18 million.

The Auditor General’s December 2025 report found significant accuracy problems. When tested with common taxpayer questions, the chatbot answered only two out of six questions correctly. The incorrect responses included wrong information about tax obligations, filing requirements, and CRA procedures — precisely the types of questions taxpayers rely on the chatbot to answer accurately.

With 18 million queries processed, the error rate identified by the Auditor General raises concerns about the accuracy of tax information provided to Canadians through the system. Taxpayers who relied on Charlie’s responses may have received incorrect information about filing requirements, deadlines, or eligibility for benefits. The CRA chatbot carries the implicit authority of the federal tax agency, and users have no straightforward way to know when the chatbot’s answer is wrong.

In November 2025, following the Auditor General’s findings, the CRA upgraded Charlie to a generative AI version and reported pre-release testing showed approximately 90% accuracy, though the agency acknowledged it could not confirm real-world accuracy. The Secretary of State for CRA pushed back against the Auditor General’s characterization of the chatbot’s performance.