In February 2024, a Canadian court ruled Air Canada legally liable for its chatbot’s false advice. The airline argued the chatbot was a “separate legal entity.” The court disagreed. The EU AI Act, finalised the same year, had already built a framework designed to prevent exactly this.
July 2026 · Case 5 of 6
As you read — hold this question
When an AI system gives advice that a customer reasonably relies on, and the advice is wrong — who is accountable?
1 ruling
that changed AI liability. Air Canada’s chatbot gave a customer false bereavement fare information. The argument that “the bot did it” failed.
Jake Moffatt asked Air Canada’s chatbot whether he could claim a bereavement discount retroactively. The chatbot said yes. Air Canada’s actual policy said no. When Moffatt applied for the refund, Air Canada refused. He took them to the Civil Resolution Tribunal. The court ruled that regardless of whether the advice came from a human agent or an automated system, Air Canada was responsible for the information it provided to customers. This is the governance question made concrete: who is accountable when AI gets it wrong?
The Framework — EU AI Act risk tiers
Quiz: Governance
Four levels of risk — and what each requires
The EU AI Act (in force August 2024, obligations phased to 2026) classifies AI systems by risk level. Higher risk means more requirements — not a ban, but more accountability, transparency, and human oversight.
Unacceptable Risk — Prohibited
Social scoring by governments. Subliminal manipulation. Real-time biometric surveillance in public (with narrow exceptions).
High Risk — Heavily regulated
Medical devices. Credit scoring. CV screening. Criminal justice. Safety-critical infrastructure. Human oversight mandatory.
Limited Risk — Transparency required
Chatbots must disclose they are AI. Deepfakes must be labelled. Emotion recognition systems must notify users.
Minimal Risk — No requirements
Spam filters. AI in video games. Most recommendation systems. No mandatory obligations — voluntary codes of practice.
Mapping Air Canada to the Act
Where the chatbot sits — and why it matters
Air Canada’s chatbot gave binding-seeming information about fare policy to a consumer making a financial decision under emotional stress. Under the EU AI Act’s framework, a customer-facing AI system giving personalised advice with financial consequences falls into the Limited Risk tier at minimum — requiring disclosure that it is AI — and potentially High Risk if integrated with booking systems.
The failure mode the Act was designed to prevent
The failure mode the Act was designed to prevent — an AI providing false information that consumers reasonably relied on — is exactly what occurred. Air Canada’s court loss established that “the bot did it” is not a legal defence. The organisation deploying the AI bears the consequences — which means the decision to deploy is a governance decision, not just a technical one.
Bias and fairness — where governance gets hard
When the training data encodes discrimination
Amazon built a hiring algorithm trained on 10 years of resumes submitted to Amazon. The company’s historical workforce was predominantly male in technical roles. The algorithm learned to penalise resumes that included the word “women’s” (as in “women’s chess club”) and downgraded graduates of all-female colleges. Amazon scrapped the system in 2018.
The bias source
Supervised learning: trained on labelled data (past resumes → hiring decisions)
Past hiring decisions encoded historical gender bias
The model learned the bias as signal, not noise
It was doing exactly what it was trained to do — that was the problem
What governance requires
Bias audits before deployment — not after harm
NYC Local Law 144 (2023): mandatory bias audits for AI hiring tools
EU AI Act: CV screening is High Risk — mandatory human review required
The principle: the data reflects the world as it was, not as it should be
The alignment question
Augmentation vs alignment — two different governance goals
Human oversight is the EU AI Act’s solution to most high-risk AI failures. But there are two very different visions of what human oversight means — and they lead to different policy choices.
Augmentation
AI assists humans. Humans retain decision authority. AI surfaces options, flags anomalies, reduces cognitive load. The doctor still diagnoses. The judge still sentences. The officer still reviews.
Risk: Automation bias — humans defer to AI recommendations even when the AI is wrong, especially under time pressure or fatigue.
Alignment
AI is designed to pursue human values and intent — not just the specified objective. RLHF is an alignment technique. The goal is AI that acts as a trustworthy agent, not one that optimises its objective at the cost of human values.
Risk: Whose values? Alignment to the developer’s preferences may not align with users’, societies’, or regulators’ needs.
The cost no one talks about
Environmental cost — the governance dimension of compute
Training a large language model emits the equivalent of 300+ tonnes of CO². A single GPT-4 query uses approximately 10× the energy of a Google search. Microsoft’s water consumption increased 34% in 2023 — largely due to AI data centre cooling. These are governance decisions, not just technical ones.
Why this belongs in governance
The EU AI Act does not currently regulate AI’s environmental impact. But the decision to deploy a foundation model rather than a smaller task-specific model is a governance choice with measurable environmental consequences. As AI scales, the compute costs scale with it. Homogenisation — the third Storey et al. property — means that every organisation using GPT-4 for routine tasks is bearing a fraction of the cost of that training run. That cost is not borne by the model. It is borne by the climate.
Take this away
Air Canada’s court loss established that “the bot did it” is not a legal defence. The organisation deploying the AI bears the consequences — which means the decision to deploy is a governance decision, not just a technical one.
What was the Air Canada ruling — and what legal principle does it establish for AI deployments?
Jake Moffatt asked Air Canada’s chatbot if he could claim a bereavement discount retroactively. The chatbot said yes. Air Canada’s policy said no. When Moffatt applied, Air Canada refused and argued the chatbot was “a separate legal entity.” The Civil Resolution Tribunal rejected this: regardless of whether the advice came from a human agent or an automated system, Air Canada was responsible for information it provided to customers. The principle: “the bot did it” is not a legal defence. The organisation deploying the AI bears the consequences.
Question 2 of 3
Map the EU AI Act’s four risk tiers — and place Air Canada’s chatbot into the correct tier.
Unacceptable Risk (prohibited): social scoring, subliminal manipulation, real-time public biometric surveillance. High Risk (heavily regulated): medical devices, credit scoring, CV screening, criminal justice — human oversight mandatory. Limited Risk (transparency required): chatbots must disclose they are AI; deepfakes must be labelled. Minimal Risk (no requirements): spam filters, AI in games, most recommendation systems. Air Canada’s chatbot: Limited Risk at minimum (customer-facing AI requiring disclosure) — potentially High Risk if integrated with booking systems giving personalised financial advice.
Question 3 of 3
Amazon’s hiring algorithm was doing exactly what it was trained to do. Explain why that was the problem — and what governance response followed.
Amazon’s algorithm was trained on 10 years of resumes submitted in an era when technical hires were predominantly male. It learned that historical hiring decisions preferred male candidates and encoded that as signal, not noise. It penalised resumes with “women’s” and downgraded all-female college graduates. The governance response: Amazon scrapped the system in 2018. NYC Local Law 144 (2023) now mandates bias audits for AI hiring tools before deployment. EU AI Act classifies CV screening as High Risk requiring mandatory human review. The principle: the data reflects the world as it was, not as it should be.
Sources
Air Canada ruling
Moffatt v Air Canada, 2024 BCCRT 149. British Columbia Civil Resolution Tribunal, February 2024.
EU AI Act
European Parliament (2024). Regulation (EU) 2024/1689 — Artificial Intelligence Act. Official Journal of the European Union.
Amazon hiring
Dastin, J. (2018, October 10). Amazon scraps secret AI recruiting tool that showed bias against women. Reuters.
NYC Local Law 144
NYC Commission on Human Rights (2023). Automated employment decision tools. nyc.gov.
Environmental cost
Luccioni, A. et al. (2023). Power hungry processing: watts driving the cost of AI deployment? arXiv:2311.16863.