Large Language Model Training Data and Canadian Privacy Rights

Foundation models are trained on data scraped from the internet including personal information of millions of Canadians — published without their knowledge, consent, or meaningful opt-out. The Office of the Privacy Commissioner of Canada and provincial counterparts have launched a joint investigation into OpenAI's ChatGPT, examining whether the company's training data practices violate Canadian privacy law and whether the generation of false biographical information about identifiable Canadians constitutes a privacy violation.

The structural challenge extends beyond any single company. Large language models embed personal information in model parameters during training in a way that makes targeted deletion technically infeasible with current methods. Traditional privacy remedies — the right to access, correct, or delete personal information — cannot be meaningfully exercised against information encoded in model weights. PIPEDA and provincial privacy legislation were designed for databases, not neural networks.

The jurisdictional dimension compounds the challenge. Foundation model training happens extraterritorially, primarily in the United States. Canadian privacy authorities can investigate and issue findings, but enforcement against foreign companies operating through cloud services requires international cooperation that current frameworks do not adequately support. This is not an edge case — it is the default condition for all Canadians whose information appears in foundation model training data.