Curriculum

AI Governance · Responsible AI

AI Ethics & Bias

AI systems can perpetuate and amplify societal biases. Fairness metrics, demographic parity constraints, and content moderation pipelines are essential guardrails for responsible deployment.

The idea

Bias enters AI systems at multiple points: biased training data, biased labels, proxy variables that correlate with protected attributes (race, gender), and historical decisions baked into labels (recidivism prediction trained on biased arrest records).

Demographic Parity: the model's positive prediction rate is equal across protected groups. Equalised Odds: true positive rate and false positive rate are equal across groups. These two objectives are often mathematically incompatible—Chouldechova's theorem proves they cannot simultaneously hold when base rates differ.

Content Moderation: multi-stage pipelines classify toxicity, hate speech, and self-harm with dedicated classifiers (Perspective API, Llama Guard, OpenAI Moderation). LLM-as-judge approaches evaluate subtler policy violations.

DP: P(Ŷ=1|A=0) = P(Ŷ=1|A=1)

Demographic Parity: the positive prediction rate must be equal across the protected attribute A.

What to hold onto

  • Counterfactual FairnessA model is counterfactually fair if its output is the same in the actual world and in a counterfactual world where the individual belonged to a different protected group, all else equal.
  • Algorithmic AuditingThird-party audits measure fairness metrics across subgroups. Required by the EU AI Act for high-risk systems. Red-teaming surfaces adversarial prompt injections and jailbreaks.
  • DocumentationModel Cards (Google) and Datasheets for Datasets (Microsoft) standardise disclosure of intended use, limitations, fairness evaluations, and known failure modes.

Live: Demographic Parity & Fairness Calculator

Adjust the classification threshold $\tau$ across demographic subgroups to inspect Equalized Odds vs Disparate Impact.

Disparate Impact: 0.92 (≥ 0.80 Rule Passed)

Quick answers

What is AI bias?

AI bias occurs when a model systematically produces unfair outcomes for specific demographic groups, typically due to biased training data, label noise, or proxy variables correlated with protected attributes. Bias is measured using fairness metrics like demographic parity and equalised odds, and mitigated through data augmentation, re-weighting, and post-processing.

What is the difference between demographic parity and equalised odds?

Demographic parity requires equal positive prediction rates across groups. Equalised odds requires equal true positive and false positive rates across groups. These two criteria are generally mutually exclusive when base rates differ between groups, a fundamental impossibility result in algorithmic fairness.