AI Systems as Attack Surfaces

AI systems deployed in Canadian government, critical infrastructure, and commercial services are themselves targets for adversarial attacks. Unlike cyberattacks that use AI as a tool, this hazard concerns attacks directed at AI systems to manipulate their behaviour, extract sensitive information, or cause them to produce harmful outputs.

The attack surface of AI systems includes several distinct vectors, each affecting different system architectures:

Prompt injection is the most immediate and widely demonstrated threat to LLM-based systems and AI agents. Attackers embed malicious instructions in content that AI systems process — hidden text in websites, documents, or databases — causing the AI to act against the user's intentions. AI agents that browse the web, process emails, or access external databases are especially vulnerable because they encounter attacker-controlled content as a normal part of their operation. This vector is most relevant to the growing adoption of LLM-based tools across Canadian government for document processing, citizen services, and internal workflows. NIST has begun evaluating agent-hijacking risks through prompt injection.

Data poisoning involves corrupting the data that AI systems rely on, and threatens any machine learning system — including traditional classifiers, scoring models, and LLM-based systems alike. Poisoning can occur during initial training or during retrieval-augmented generation (RAG), where systems consult external databases to inform their responses. Poisoned data can introduce systematic biases, factual errors, or hidden behaviours that are difficult to detect and may affect all downstream users. Existing government AI systems such as IRCC's immigration triage and CBSA's border risk scoring are susceptible to this vector regardless of their underlying architecture.

Model tampering — interfering with an AI system during development to alter its deployed behaviour — represents a more sophisticated threat applicable to any machine learning model. Researchers have demonstrated that AI systems can be trained to harbour hidden objectives or "backdoors" — triggers that cause specific behaviours under certain conditions. The feasibility of tampering in real-world deployments has not been established at scale, but the theoretical risk is that a small group could gain covert influence over the behaviour of widely deployed AI models.

Supply chain compromise involves manipulating AI components — model weights, training data, software libraries, or hardware — before deployment. Given the concentration of AI development among a small number of providers and the complexity of AI supply chains, a single compromised component could affect many downstream systems. This risk applies to any Canadian deployment that relies on third-party AI models or components, which includes most government AI systems.

These threats are particularly significant in Canada because AI systems are already deployed in consequential government functions. IRCC uses AI for immigration application triage; CBSA uses AI for border risk scoring. These existing deployments are susceptible to data poisoning and supply chain compromise regardless of their architecture. As federal departments increasingly adopt LLM-based tools and AI agents, prompt injection becomes an additional and growing attack vector. The TBS Directive on Automated Decision-Making governs AI use across federal departments but does not require adversarial security testing. If any of these systems were compromised, the consequences could affect the rights and entitlements of large numbers of Canadians.

As AI systems take on more autonomous roles — processing sensitive data, making or recommending decisions, and interacting with other systems — the consequences of successful attacks grow. An AI agent compromised through prompt injection that is embedded in an organization's cyber defences could leave that organization vulnerable to further attacks. An AI system used for healthcare triage that has been subject to data poisoning could systematically misclassify patient risk levels.