⚖️ Bias in AI

The Mirror Analogy

A mirror reflects exactly what's in front of it - flaws and all. Put on a crooked tie, the mirror shows a crooked tie.

AI is like a mirror for data.

If the training data contains biases, the AI learns and reflects those biases. Show it historical hiring decisions that favored men? It learns to favor men. It's not malicious - it's simply learning patterns from what you gave it.

The scary part: AI doesn't just reflect bias, it can amplify and scale it to millions of decisions.

Why AI Bias Matters

Scale of Impact

A single biased human might make a limited number of hiring decisions. A biased AI can influence decisions at much larger scale.

Hidden Nature

Human bias is visible - you can question a person's reasoning. AI bias is hidden in mathematical patterns - hard to see, hard to challenge.

Trust and Legitimacy

People assume "the computer said so" means it's objective. But the computer learned from biased human decisions.

How Bias Gets Into AI

The Pipeline

Real World (contains human bias)
    ↓
Data Collection (captures that bias)
    ↓
Training (model learns biased patterns)
    ↓
Deployment (bias applied at scale)
    ↓
Feedback Loop (biased outputs create more biased data)

Specific Sources

Real-World Examples

Hiring Resume Screening

What happened: A company built an AI to review resumes and identify good candidates.

The bias: It penalized resumes that mentioned "women's" (like "women's chess club" or "women's college").

Why: The AI was trained on many past hiring decisions where men were predominantly hired. It learned a shortcut: “male-associated words = good.”

Outcome: Amazon scrapped the tool.

Face Recognition Performance Gaps

What happened: Researchers tested commercial face recognition systems.

The bias: The system made noticeably more mistakes for some groups (like darker-skinned women) than others (like lighter-skinned men).

Why: Training datasets contained far more light-skinned faces than dark-skinned faces. The AI literally hadn't seen enough examples to learn.

Healthcare Prediction

What happened: An AI predicted which patients needed extra care.

The bias: It systematically recommended less care for Black patients.

Why: It used healthcare spending as a proxy for health needs. But Black patients historically had less access to healthcare, so they spent less. The AI confused "can't access care" with "doesn't need care."

Sentencing Risk Scores

What happened: COMPAS algorithm predicted criminal reoffending risk.

The bias: Falsely flagged Black defendants as high-risk at twice the rate of white defendants.

Why: Historical arrest data reflected discriminatory policing patterns.

Types of AI Bias

1. Historical Bias

Data reflects past discrimination:

Hiring data: "Mostly men were promoted"
AI learns: "Men are better candidates"

2. Representation Bias

Some groups underrepresented:

Training data: Mostly photos of younger people
AI: Works well for younger faces, struggles more with older faces

3. Measurement Bias

What you measure introduces bias:

Using arrests as proxy for crime
But arrests reflect policing patterns, not actual crime rates

4. Aggregation Bias

Using one model across many groups can miss important group differences:

Diabetes model designed for one population
Applied to another population with different risk factors

5. Deployment Bias

Model used inappropriately:

AI designed for one context
Deployed in a completely different context

How to Spot Bias

Disaggregate Performance Metrics

Don't just look at a single overall score:

Overall score: looks “good”
Group A: performs better
Group B: performs worse
← The gap can be hidden by a single overall metric.

Check Disparate Impact

Are outcomes different for different groups?

Loan approvals:
- Group A: approved more often
- Group B: approved less often
← A big disparity can suggest possible bias

Examine Training Data

Who's in the data? Who's missing?

How to Reduce Bias

1. Diverse, Representative Training Data

Collect data that represents all groups fairly:

Before: One group is overrepresented
After: Coverage improves across groups (or matches the real population)

2. Bias Auditing

Test model performance across demographics before deployment.

3. Fairness Constraints

Build fairness into the optimization:

Regular training: Maximize accuracy
Fairness-aware training: Maximize accuracy WHILE maintaining equal error rates across groups

4. Regular Monitoring

Bias can emerge over time as data shifts. Continuous monitoring catches drift.

5. Human Oversight

Keep humans in the loop for high-stakes decisions:

AI: Recommends denial
Human: Reviews and can override

The Uncomfortable Truth

You cannot create a fully fair AI because:

1. "Fairness" has multiple, sometimes contradicting definitions
2. Historical data is inherently biased
3. Stakeholders have different views on what's fair

The goal isn't perfection - it's awareness, measurement, and continuous improvement.

FAQ

Q: Can we eliminate bias completely?

No. We can reduce, manage, and mitigate it. Bias is an ongoing concern, not a one-time fix.

Q: Who is responsible for AI bias?

Many people: data collectors, model builders, product managers, executives, and regulators. It’s a shared responsibility.

Q: What is fairness in machine learning?

Many competing definitions:

• Demographic parity: Equal positive outcomes across groups
• Equalized odds: Equal accuracy across groups
• Individual fairness: Similar people treated similarly

No single definition works for all contexts.

Q: Is biased AI illegal?

Depends on jurisdiction and use case. In some areas (lending, employment), discriminatory AI can violate civil rights laws.

Q: How do I know if my AI is biased?

Disaggregate metrics by group. Compare performance across demographics. Audit regularly.

Q: Can't we just remove sensitive features?

Removing race, gender, etc. doesn't remove bias. Other features (zip code, shopping patterns) can proxy for protected attributes.

Summary

AI Bias occurs when systems learn unfair patterns from biased data. It can cause real harm at scale in hiring, lending, healthcare, and justice. Mitigation requires diverse data, auditing, fairness constraints, and ongoing monitoring.

Key Takeaways:

• AI reflects and amplifies biases in training data
• Sources: historical data, representation gaps, proxy variables
• Real impacts: hiring, facial recognition, healthcare, sentencing
• No single definition of "fairness" - context matters
• Mitigation: diverse data, auditing, monitoring, human oversight
• Not a one-time fix - requires ongoing attention

Building fair AI isn't just good ethics - it's essential for building systems people can trust.