33.1. Bias Detection: Engineering Fairness

Important

The Engineering Reality: Fairness is not a “soft skill.” It is a mathematical constraint. If your model’s False Positive Rate for Group A is 5% and for Group B is 25%, you have built a discriminatory machine. This section details how to detect, measure, and mitigate outcome disparities in production systems.

Bias in Machine Learning is often treated as a PR problem. In MLOps, we treat it as a System Defect, equivalent to a memory leak or a null pointer exception. We can define it, measure it, and block it in CI/CD.

33.1.1. The Taxonomy of Bias

Before we write code, we must understand what we are chasing.

33.1.2. The Metrics of Fairness

There is no single definition of “Fair.” You must choose the metric that matches your legal and ethical constraints.

1. Disparate Impact Ratio (DIR)

• Definition: The ratio of the selection rate of the protected group to the reference group.
• Formula: $P(\hat{Y}=1 | A=minority) / P(\hat{Y}=1 | A=majority)$
• Threshold: The Four-Fifths Rule (80%). If DIR < 0.8, it is likely illegal employment discrimination in the US.

2. Equal Opportunity (TPR Parity)

• Definition: True Positive Rates should be equal across groups.
• Scenario: “If a person is actually qualified for the loan, they should have the same probability of being approved, regardless of gender.”
• Formula: $P(\hat{Y}=1 | Y=1, A=0) = P(\hat{Y}=1 | Y=1, A=1)$

3. Predictive Parity (Precision Parity)

• Definition: If the model predicts “High Risk,” the probability of being truly High Risk should be the same.
• Scenario: Recidivism prediction (COMPAS). A score of “8” should mean “60% risk of re-offense” for both Black and White defendants.

33.1.3. Tooling: Fairlearn Deep Dive

Fairlearn is the industry standard Python library for assessment and mitigation.

Implementing a Bias Dashboard:

import pandas as pd
from fairlearn.metrics import MetricFrame, selection_rate, false_positive_rate
from sklearn.metrics import accuracy_score
from sklearn.ensemble import RandomForestClassifier

# 1. Load Data
# Data usually contains: Features (X), Labels (Y), and Sensitive Attributes (A)
df = pd.read_csv("loan_data_clean.csv")
X = df.drop(columns=["default", "gender", "race"])
y = df["default"]
A_gender = df["gender"] # Sensitive Attribute

# 2. Train a naive model
model = RandomForestClassifier()
model.fit(X, y)
y_pred = model.predict(X)

# 3. Create the MetricFrame
# This is the core Fairlearn object that groups metrics by sensitive attribute
metrics = {
    "accuracy": accuracy_score,
    "selection_rate": selection_rate,
    "false_positive_rate": false_positive_rate
}

mf = MetricFrame(
    metrics=metrics,
    y_true=y,
    y_pred=y_pred,
    sensitive_features=A_gender
)

# 4. Analysis
print("Overall Metrics:")
print(mf.overall)

print("\nMetrics by Group:")
print(mf.by_group)

# 5. Check Disparate Impact
# Extract selection rates
sr_male = mf.by_group.loc["Male", "selection_rate"]
sr_female = mf.by_group.loc["Female", "selection_rate"]

dir_score = sr_female / sr_male
print(f"\nDisparate Impact Ratio (Female/Male): {dir_score:.4f}")

if dir_score < 0.8:
    print("[FAIL] Four-Fifths Rule Violated.")

33.1.4. Mitigation Strategies

If you detect bias, you have three implementation points to fix it.

Pre-Processing (Reweighing)

Modify the training data weights so that the loss function pays more attention to the minority group.

• Tool: fairlearn.preprocessing.CorrelationRemover (Linear decorrelation of features).

In-Processing (Adversarial Debiasing)

Add a “Fairness Constraint” to the optimization problem. Minimize $Loss(Y, \hat{Y})$ subject to $Correlation(\hat{Y}, A) < \epsilon$.

• Tool: fairlearn.reductions.ExponentiatedGradient. This treats fairness as a constrained optimization problem and finds the Pareto frontier.

from fairlearn.reductions import ExponentiatedGradient, DemographicParity

# Define the constraint: Demographic Parity (Equal Selection Rates)
constraint = DemographicParity()

# Wrap the base model
mitigator = ExponentiatedGradient(
    estimator=RandomForestClassifier(),
    constraints=constraint
)

mitigator.fit(X, y, sensitive_features=A_gender)
y_pred_mitigated = mitigator.predict(X)

Post-Processing (Threshold Adjustment)

Train the model blindly (naive). Then, during inference, use different thresholds for different groups to achieve equity.

• Warning: This is explicit affirmative action and may be illegal in certain jurisdictions (e.g., California prop 209). Consult legal.

33.1.5. CI/CD Architecture: The Fairness Gate

You cannot rely on notebooks. Bias detection must be automated in the pipeline.

Architecture:

1. Pull Request: DS opens a PR with new model code.
2. CI Build:
   • Train Candidate Model.
   • Load Golden Validation Set (Must contain Sensitive Attributes).
   • Run bias_audit.py.
3. Gate:
   • If DIR < 0.8, fail the build.
   • If Accuracy drop > 5% compared to Main, fail the build.

GitHub Actions Implementation:

name: Fairness Audit
on: [pull_request]

jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install dependencies
        run: pip install fairlearn pandas scikit-learn
      - name: Run Audit
        run: python scripts/audit_model.py --model candidate.pkl --data validation_sensitive.parquet --threshold 0.8
        continue-on-error: false

33.1.6. Monitoring Bias in Production (Drift)

Bias is not static. If your user demographic shifts, your bias metrics shift.

• Scenario: You trained on US data (DIR=0.9). You launch in India. The model features behave differently. DIR drops to 0.4.

The Monitoring Loop:

1. Inference Logger: Logs inputs, outputs.
2. Attribute Joiner: Crucial Step. The inference logs rarely contain “Gender” or “Race” (we don’t ask for it at runtime). You must join these logs with your Data Warehouse (Offline) to recover the sensitive attributes for analysis.
   • Note: This requires strict PII controls.
3. Calculator: Daily batch job computes DIR on the joined data.
4. Alert: If DIR drops below threshold, page the Responsible AI team.

33.1.7. Summary

Bias is an engineering defect.

1. Measure: Use Fairlearn MetricFrame to disaggregate metrics.
2. Gate: Block biased models in CI/CD.
3. Monitor: Re-calculate fairness metrics in production daily.
4. Mitigate: Use algorithmic debiasing (ExponentiatedGradient) rather than just “removing columns.”

[Previous content preserved…]

33.1.8. Deep Dive: IBM AIF360 vs. Microsoft Fairlearn

You have two main heavyweights in the open-source arena. Which one should you use?

IBM AIF360 (AI Fairness 360)

• Philosophy: “Kitchen Sink.” It implements every metric and algorithm from academia (70+ metrics).
• Pros: Extremely comprehensive. Good for research comparisons.
• Cons: Steep learning curve. The API is verbose. Hard to put into a tight CI/CD loop.
• Best For: The “Center of Excellence” team building broad policies.

Microsoft Fairlearn

• Philosophy: “Reductionist.” It reduces fairness to an optimization constraint.
• Pros: Scikit-learn compatible style (fit/predict). Very fast. Easy to explain to engineers.
• Cons: Fewer algorithms than AIF360.
• Best For: The MLOps Engineer trying to block a deploy in Jenkins.

Recommendation: Start with Fairlearn for the pipeline. Use AIF360 for the quarterly deep audit.

33.1.9. Calibration vs. Equal Opportunity (The Impossibility Theorem)

A critical mathematical reality: You cannot satisfy all fairness metrics simultaneously.

Kleinberg’s Impossibility Theorem proves that unless base rates are equal (which they rarely are), you cannot satisfy:

1. Calibration (Precision Parity).
2. Equalized Odds (TPR/FPR Parity).
3. Balance for the Negative Class.

The Engineering Choice: You must choose ONE worldview based on the harm.

• Punitive Harm (Jail/Loan Denial): Use Equal Opportunity. You do not want to unjustly punish a qualified minority.
• Assistive Harm (Job Ad/Coupon): Use Calibration. You want the “score” to mean the same thing for everyone effectively.

33.1.10. Explainability as a Bias Detector (SHAP)

Sometimes metrics don’t tell the story. You need to see why the model is racist. SHAP (SHapley Additive exPlanations) decomposes the prediction into feature contributions.

Detecting Proxy Variables: If Zip_Code has a higher SHAP value than Income for loan denial, your model has likely learned “Zip Code” is a proxy for “Race.”

Python Implementation:

import shap

# 1. Train
model = xgboost.train(params, dtrain)

# 2. Explain
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X)

# 3. Stratify by Group
# Compare average SHAP values for 'Zip_Code' between Groups
group_A_idx = df['race'] == 0
group_B_idx = df['race'] == 1

impact_A = np.abs(shap_values[group_A_idx]).mean(axis=0)
impact_B = np.abs(shap_values[group_B_idx]).mean(axis=0)

print(f"Feature Importance for Group A: {impact_A}")
print(f"Feature Importance for Group B: {impact_B}")

# If Feature 5 (Zip) is 0.8 for Group A and 0.1 for Group B, 
# the model relies on Zip Code ONLY to penalize Group A.

33.1.11. Legal Landscape: Disparate Treatment vs. Disparate Impact

Engineers need to speak “Legal” to survive the compliance review.

1. Disparate Treatment (Intentional)

• Definition: Explicitly checking if race == 'Black': deny().
• Engineering Equivalent: Including “Race” or “Gender” as explicit features in X_train.
• Control: drop_columns is not enough (because of proxies), but it is the bare minimum.

2. Disparate Impact (Unintentional)

• Definition: A facially neutral policy (e.g., “Must be 6ft tall”) that disproportionately affects a protected group (Women).
• Engineering Equivalent: Training on “Height” which correlates with “Gender.”
• Defense: “Business Necessity.” You must prove that Height is strictly necessary for the job (e.g., NBA Player), not just “helpful.”

33.1.12. Case Study: The Healthcare Algorithm (Science 2019)

The Failure: A widely used algorithm predicted “Health Risk” to allocate extra care. The Proxy: It used “Total Healthcare Cost” as the target variable ($Y$). The Bias: Black patients have less access to care, so they spend less money than White patients for the same sickness level. The Result: The model rated Black patients as “Lower Risk” (because they cost less), denying them care. The Fix: Change $Y$ from “Cost” to “Critical Biomarkers.”

Lesson: Bias is usually in the Label ($Y$), not just the Features ($X$).

33.1.13. Automated Documentation (Datasheets as Code)

We can generate a PDF report for every training run.

from jinja2 import Template
import matplotlib.pyplot as plt

def generate_fairness_report(metrics_dict, run_id):
    # 1. Plot
    metrics_dict.by_group.plot(kind='bar')
    plt.title(f"Fairness Metrics Run {run_id}")
    plt.savefig("fairness.png")
    
    # 2. Render HTML
    html = """
    <h1>Fairness Audit: Run {{ run_id }}</h1>
    <h2>Disparate Impact Ratio: {{ dir }}</h2>
    <img src="fairness.png">
    {% if dir < 0.8 %}
    <h3 style="color:red">FAIL: ADJUSTMENT REQUIRED</h3>
    {% else %}
    <h3 style="color:green">PASS</h3>
    {% endif %}
    """
    
    # 3. Save to S3
    s3.upload("report.html", f"reports/{run_id}.html")

[End of Section 33.1]