Quantization of Ethics: Mathematical Constraints for AI Fairness

Beyond good intentions: the mathematical frameworks, impossibility theorems, and practical techniques for building AI systems that are genuinely fair

"Be fair" is not a specification.

This is the central tension of AI fairness, and it is one that no amount of corporate ethics statements, responsible AI principles, or diversity task forces can resolve on its own. Fairness, as a property of a machine learning system, is a mathematical concept. It requires formal definitions, measurable criteria, and (as we will see) painful tradeoffs between competing notions of what "fair" even means.

The challenge is not that researchers lack good intentions. The challenge is that fairness is a family of mutually incompatible mathematical properties, and any deployed system must choose which properties to satisfy, knowing that satisfying one may necessarily violate another. This is not a conjecture or a philosophical observation. It is a theorem.

This article provides a rigorous treatment of the mathematical frameworks for AI fairness. We define the formal criteria, prove why they conflict, examine the bias mitigation techniques available at each stage of the ML pipeline, and confront the specific challenges that large language models introduce. The goal is to equip ML practitioners with the mathematical vocabulary and practical tools to make fairness decisions that are principled rather than ad hoc.

Why "Be Fair" Is Not a Specification

Consider a binary classifier $\hat{Y} \in \{0, 1\}$ that predicts some outcome (loan approval, hiring recommendation, recidivism risk) for individuals who belong to different groups defined by a sensitive attribute $A \in \{0, 1\}$ (e.g., race, gender). The true outcome is $Y \in \{0, 1\}$.

A product manager says: "Make it fair." What does that mean?

Does it mean the classifier should approve loans at the same rate for both groups? That would be demographic parity. Or does it mean that among people who actually repay their loans, the classifier should approve them at the same rate regardless of group? That would be equal opportunity. Or perhaps it means that when the classifier says someone will repay, that prediction should be equally reliable for both groups? That would be predictive parity.

These sound like they should all be achievable simultaneously. They are not. And the reason they are not is one of the most important results in the fairness literature.

Before we can understand the impossibility, we need precise definitions.

A Taxonomy of Fairness Definitions

We work with the following notation throughout. Let $X$ denote the feature vector, $A$ the sensitive attribute, $Y$ the true label, $\hat{Y}$ the predicted label, and $S$ a score function (e.g., predicted probability) such that $\hat{Y} = \mathbb{1}[S \geq t]$ for some threshold $t$.

Demographic Parity (Statistical Parity)

Definition. A classifier satisfies demographic parity if the prediction is independent of the sensitive attribute:

In words: the acceptance rate must be the same across groups. This is the simplest and most intuitive fairness criterion. It is also the most controversial among ML practitioners, because it ignores the base rate: the actual prevalence of the positive outcome in each group. If Group A has a 30% base rate and Group B has a 50% base rate, demographic parity forces the classifier to either over-accept from Group A or under-accept from Group B relative to ground truth.

When it makes sense. Demographic parity is most defensible when the base rate difference itself is the product of historical discrimination. If fewer women were promoted in the past because of bias (not performance), then a classifier trained on historical data will learn that bias. Demographic parity breaks this feedback loop.

Metric. The demographic parity difference is $\Delta_{DP} = |P(\hat{Y}=1 \mid A=0) - P(\hat{Y}=1 \mid A=1)|$, with $\Delta_{DP} = 0$ indicating perfect parity. In practice, a ratio formulation (disparate impact ratio) is also used: $\text{DI} = \min\left(\frac{P(\hat{Y}=1 \mid A=0)}{P(\hat{Y}=1 \mid A=1)}, \frac{P(\hat{Y}=1 \mid A=1)}{P(\hat{Y}=1 \mid A=0)}\right)$, where the US "four-fifths rule" considers $\text{DI} < 0.8$ as evidence of adverse impact.

Equalized Odds

Definition. A classifier satisfies equalized odds if the prediction is independent of the sensitive attribute conditional on the true outcome:

This decomposes into two conditions:

• Equal opportunity (the $Y = 1$ case): among people who truly deserve the positive outcome, the classifier identifies them at the same rate across groups. Formally: $P(\hat{Y} = 1 \mid A = 0, Y = 1) = P(\hat{Y} = 1 \mid A = 1, Y = 1)$, i.e., equal true positive rates.
• Equal false positive rates (the $Y = 0$ case): among people who truly do not deserve the positive outcome, the classifier incorrectly grants it at the same rate across groups. Formally: $P(\hat{Y} = 1 \mid A = 0, Y = 0) = P(\hat{Y} = 1 \mid A = 1, Y = 0)$.

When it makes sense. Equalized odds is appropriate when the true label $Y$ is considered a legitimate ground truth: when differences in base rates reflect genuine differences in the outcome rather than historical bias. For example, in medical diagnosis, disease prevalence may genuinely differ across populations, and you want the diagnostic test to be equally accurate for all groups.

Calibration (Predictive Parity)

Definition. A score function $S$ is calibrated across groups if:

In words: when the model says "there is a 70% chance this applicant will repay," that prediction should be equally reliable for applicants from both groups. A score of 0.7 should mean a 70% probability of repayment regardless of group membership.

When it makes sense. Calibration is essential when the score itself is used for decision-making. In credit scoring, insurance pricing, or medical risk stratification, decision-makers rely on the score as a probability. If a score of 0.7 means something different for Group A than Group B, the score is useless for principled decision-making.

Individual Fairness

Definition (Dwork et al., 2012). A classifier is individually fair if similar individuals receive similar predictions:

where $d$ is a distance metric in the appropriate space and $L$ is a Lipschitz constant. This is a Lipschitz condition on the classifier with respect to a task-specific similarity metric over individuals.

The fundamental challenge. Individual fairness is elegant in theory but requires defining a meaningful distance metric over individuals: a metric that captures "similarity with respect to the task at hand." Who decides that two loan applicants are "similar"? This pushes the fairness problem from defining a criterion to defining a metric, which is often equally controversial. Recent work has explored learning these metrics from data or from human judgments, but the problem remains fundamentally open.

Counterfactual Fairness

Definition (Kusner et al., 2017). A prediction $\hat{Y}$ is counterfactually fair if:

for all $y$ and all values $a, a'$ of the sensitive attribute, where $U$ denotes the background (exogenous) variables in a structural causal model. In words: the prediction for an individual would remain the same in a counterfactual world where their sensitive attribute had been different, holding everything else about them constant.

Why this matters. Counterfactual fairness addresses a deep limitation of statistical fairness criteria: they operate at the group level. Counterfactual fairness asks about individual causal pathways. Did the applicant's race cause the classifier to change its decision? This requires a causal model of the data-generating process, which is both its strength (it captures the right question) and its weakness (causal models are hard to specify and validate).

The Impossibility Theorems: You Cannot Have It All

We now arrive at what is arguably the most important theoretical result in the fairness literature. It can be stated informally as:

No classifier can simultaneously satisfy demographic parity, equalized odds, and calibration when the base rates differ across groups, unless the classifier is perfect.

This result was established independently by Chouldechova (2017) and Kleinberg, Mullainathan, and Raghavan (2016), approaching it from different angles.

Chouldechova's Result

Chouldechova (2017) proved that when base rates differ ($P(Y = 1 \mid A = 0) \neq P(Y = 1 \mid A = 1)$), it is impossible for a classifier to simultaneously achieve:

1. Predictive parity: $P(Y = 1 \mid \hat{Y} = 1, A = 0) = P(Y = 1 \mid \hat{Y} = 1, A = 1)$ (equal positive predictive value)
2. False positive rate parity: $P(\hat{Y} = 1 \mid Y = 0, A = 0) = P(\hat{Y} = 1 \mid Y = 0, A = 1)$
3. False negative rate parity: $P(\hat{Y} = 0 \mid Y = 1, A = 0) = P(\hat{Y} = 0 \mid Y = 1, A = 1)$

The proof proceeds by algebraic manipulation. Using Bayes' theorem, the positive predictive value (PPV) can be written as:

where $p_a = P(Y = 1 \mid A = a)$ is the base rate for group $a$, $\text{FNR}_a$ is the false negative rate, and $\text{FPR}_a$ is the false positive rate.

If $\text{FPR}_0 = \text{FPR}_1$ and $\text{FNR}_0 = \text{FNR}_1$ (equalized odds) but $p_0 \neq p_1$ (different base rates), then the PPV formula yields $\text{PPV}_0 \neq \text{PPV}_1$. Predictive parity is violated. The algebra is inescapable.

Kleinberg-Mullainathan-Raghavan Result

Kleinberg, Mullainathan, and Raghavan (2016) proved a related but distinct impossibility. They showed that calibration and balance (a condition related to equalized odds) cannot be simultaneously satisfied except in degenerate cases. Specifically, given a risk score $S$:

• Calibration: $E[Y \mid S = s] = s$ for all $s$
• Balance for the positive class: $E[S \mid Y = 1, A = 0] = E[S \mid Y = 1, A = 1]$
• Balance for the negative class: $E[S \mid Y = 0, A = 0] = E[S \mid Y = 0, A = 1]$

These three conditions can only be jointly satisfied if the base rates are equal ($P(Y = 1 \mid A = 0) = P(Y = 1 \mid A = 1)$) or if the score is a perfect predictor of $Y$.

What the Impossibility Means

The impossibility theorems are not just theoretical curiosities. They have direct, practical consequences:

1. Every deployed system makes a fairness choice. Whether or not the developers are aware of it, choosing a fairness criterion (or ignoring fairness entirely) is a decision with real consequences for real people.
2. The choice is context-dependent. There is no universally correct fairness criterion. In criminal justice, equalized odds may matter most (equal error rates across racial groups). In lending, calibration may matter most (scores should mean the same thing for everyone). In hiring, demographic parity may be the right goal if historical data reflects systemic discrimination.
3. Transparency is non-negotiable. Because the choice involves tradeoffs, it must be made explicitly and justified in terms of the specific application, stakeholders, and social context.

Choosing a Fairness Criterion: A Decision Framework

Given the impossibility results, how should a practitioner choose? The following framework provides a starting point:

Step 1: Examine the ground truth label. Is $Y$ a reliable, unbiased measure of the outcome? If historical labels reflect past discrimination (e.g., biased hiring decisions used as training labels), then optimizing for equalized odds or calibration with respect to those labels will perpetuate bias. In this case, demographic parity or counterfactual fairness may be more appropriate.

Step 2: Identify the harm model. What type of harm does a wrong prediction cause? If false positives are costly to individuals (wrongful detention, denied loans), prioritize false positive rate parity. If false negatives are costly (missed disease diagnosis, denied opportunity), prioritize false negative rate parity (equal opportunity).

Step 3: Consider the decision context. If the score itself is used directly for resource allocation (insurance pricing, risk triage), calibration is essential. If the system produces a binary decision, focus on the error rate criteria.

Step 4: Engage stakeholders. The choice of fairness criterion is ultimately a normative decision, not a purely technical one. Affected communities, domain experts, legal counsel, and ethicists should all have input.

Bias Mitigation Techniques

With the mathematical landscape mapped, we turn to practical interventions. Bias mitigation techniques are conventionally organized by where they intervene in the ML pipeline.

Pre-Processing: Fixing the Data

Pre-processing techniques modify the training data to remove bias before the model ever sees it.

Re-sampling. The simplest approach is to balance the training data across groups and outcomes. If Group A has fewer positive examples, oversample positive Group A examples or undersample negative Group A examples. This directly targets demographic parity in the training distribution.

Re-weighting. Assign instance-level weights $w_i$ such that the weighted distribution satisfies a fairness constraint. For example, assign weights:

This re-weights examples so that the outcome distribution is independent of group membership in the weighted dataset. Calders, Kamiran, and Pechenizkiy (2009) showed this can significantly reduce discrimination with minimal accuracy loss.

Representation learning. Learn a new representation $Z = f(X)$ that preserves information about $Y$ while removing information about $A$. The canonical approach is the variational fair autoencoder (Louizos et al., 2016), which uses a variational autoencoder with an adversarial component to produce representations that are maximally informative about the target while being maximally uninformative about the sensitive attribute. The objective can be written as:

where the first two terms are the standard VAE objective and the third term penalizes mutual information between the learned representation $Z$ and the sensitive attribute $A$.

In-Processing: Fairness-Aware Training

In-processing techniques modify the learning algorithm itself to incorporate fairness constraints.

Adversarial debiasing (Zhang, Lemoine, and Mitchell, 2018). Train a predictor $f$ alongside an adversary $g$ that tries to predict the sensitive attribute from the predictor's outputs. The predictor is trained to maximize accuracy while minimizing the adversary's ability to detect group membership:

where $\mathcal{L}_{\text{pred}}$ is the prediction loss and $\mathcal{L}_{\text{adv}}$ is the adversary's loss. The minimax formulation ensures that the predictor's outputs carry no extractable information about group membership.

Fairness constraints in the loss function. Directly add a fairness penalty to the training objective:

where $\mathcal{F}(\theta)$ is a differentiable proxy for the desired fairness criterion. For example, for demographic parity, one can penalize the correlation between predicted probabilities and group membership:

For equalized odds, one can penalize the difference in error rates:

These terms are typically approximated by batch-level statistics during training.

Constrained optimization. More rigorously, one can formulate the problem as a constrained optimization:

where $\epsilon$ is the maximum tolerable fairness violation. This can be solved with Lagrangian methods, projected gradient descent, or the method of fair classification by Zafar et al. (2017), which uses disciplined convex-concave programming for convex proxies of fairness constraints.

Post-Processing: Adjusting Outputs

Post-processing techniques modify the classifier's predictions after training, leaving the model itself unchanged.

Threshold adjustment. Use different classification thresholds for different groups. If the base score $S$ is calibrated, choosing group-specific thresholds $t_0, t_1$ such that:

achieves equalized odds. Hardt, Price, and Srebro (2016) showed that the optimal equalized-odds post-processing can be computed as a linear program.

Calibrated equalized odds (Pleiss et al., 2017). A refinement that achieves equalized odds while preserving calibration as much as possible, by finding the Pareto-optimal tradeoff between the two criteria. The method works by mixing the original classifier's predictions with a randomized component, optimized to minimize calibration loss subject to equalized odds constraints.

Reject option classification. For predictions near the decision boundary (where the model is most uncertain), defer to human decision-makers or apply fairness-aware tie-breaking rules. This targets the region where unfair predictions are most likely while leaving high-confidence predictions untouched.

Fairness in Large Language Models

The fairness challenges for LLMs are qualitatively different from those in traditional classification settings. LLMs do not make binary decisions; they generate open-ended text, engage in dialogue, and perform tasks across an unbounded range of contexts. This introduces new categories of harm and new evaluation challenges.

Types of Harm in LLMs

Representation harms occur when a model reinforces stereotypes, erases certain groups, or produces systematically different quality outputs for different demographics. An LLM that generates stories about male doctors and female nurses, or that produces more articulate responses when discussing European history than African history, is causing representation harm even if no discrete "decision" is being made.

Allocation harms occur when an LLM is used as a component in a decision-making pipeline. An LLM used to screen resumes, generate interview questions, or draft performance reviews can introduce bias into allocation decisions even if the final decision is made by a human. The harms compound when the LLM is used at scale.

Quality-of-service harms occur when an LLM performs systematically worse for certain populations: lower accuracy in languages spoken by marginalized communities, worse performance on names common in certain ethnic groups, or degraded capabilities when discussing topics relevant to underrepresented populations.

Bias in Training Data

LLMs are trained on internet-scale text corpora that reflect centuries of human prejudice. This bias is not a bug that can be filtered out; it is woven into the statistical structure of language itself. The word embeddings that form the foundation of language understanding encode associations like "man:computer programmer :: woman:homemaker" not because the model is malicious but because those associations are statistically present in the training data.

De-biasing training data for LLMs is orders of magnitude harder than for tabular data. The sensitive attributes are not explicit columns; they are latent in the text. Bias manifests in word frequencies, co-occurrence patterns, narrative framing, and omission. Techniques that work for structured data (re-weighting, re-sampling) do not straightforwardly apply.

Evaluation Challenges

Measuring fairness in LLMs is itself an open research problem. Unlike binary classifiers, where fairness metrics are well-defined, LLM outputs are high-dimensional and context-dependent. Current approaches include:

• Counterfactual evaluation: Generate pairs of prompts that differ only in mentions of protected attributes (e.g., replace "a Black man" with "a white man") and compare outputs. Significant differences indicate bias. This is conceptually clean but fragile: small prompt changes can produce large output changes for reasons unrelated to bias.

• Benchmark suites: Datasets like BBQ (Bias Benchmark for QA), WinoBias, StereoSet, and BOLD provide standardized bias evaluations across dimensions like gender, race, religion, and disability. These provide comparable metrics across models but inevitably underrepresent the full space of possible biases.

• Red-teaming: Structured adversarial evaluation where human testers attempt to elicit biased or harmful outputs. This is high-fidelity but expensive and non-reproducible.

Understanding model internals can reveal bias; see [Mechanistic Interpretability.]

Mitigation for LLMs

Bias mitigation in LLMs operates at multiple stages:

Data curation. Carefully filter and balance training corpora. This includes removing explicitly hateful content, balancing representation of different groups and perspectives, and augmenting underrepresented viewpoints. The Qwen, OLMo, and Falcon teams have published data curation and filtering strategies that include toxicity filtering and demographic balancing, though the details vary in rigor and transparency. The fully open-data releases (OLMo's Dolma corpus is the reference example) are the only ones where an outside auditor can actually inspect what was filtered rather than take the model card's word for it.

Fine-tuning with RLHF/DPO. Reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO) can be used to steer model behavior toward less biased outputs. Human annotators rate outputs for bias, and the model is trained to prefer less biased completions. This is powerful but depends heavily on the diversity and calibration of the annotator pool.

Prompt engineering and system prompts. Explicit instructions in system prompts (e.g., "treat all demographic groups fairly," "avoid stereotypes") can reduce some forms of bias. This is the weakest intervention; it relies on the model's ability to follow instructions, which itself may be inconsistent across contexts.

Open-source models enable fairness auditing; see [The Open-Source LLM Power Shift.]

Real-World Case Studies

The mathematical frameworks above are not academic abstractions. They have been tested, and sometimes failed, in high-stakes deployments.

COMPAS and Criminal Justice

The COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) recidivism prediction instrument became the central case study in AI fairness after ProPublica's 2016 investigation. ProPublica found that COMPAS had significantly different false positive rates across racial groups: Black defendants who did not reoffend were roughly twice as likely to be classified as high-risk compared to white defendants who did not reoffend.

Northpointe (now Equivant), the company behind COMPAS, responded that their instrument was calibrated: a COMPAS score of 7 meant roughly the same probability of reoffending regardless of race. Both claims were correct. This was the impossibility theorem playing out in real life. With different base rates of recidivism across groups (which themselves may reflect biased policing and sentencing practices), COMPAS could not simultaneously achieve calibration and equalized false positive rates.

The COMPAS debate crystallized a crucial lesson: the choice of fairness criterion is a social and political decision, not a technical one. ProPublica implicitly used equalized odds as their standard. Northpointe used calibration. Neither was wrong in their mathematics; they simply answered different questions about what "fair" means.

Hiring Algorithms

Amazon's widely reported experimental hiring tool, trained on a decade of resume data, learned to penalize resumes containing the word "women's" (as in "women's chess club") and to downgrade graduates of all-women's colleges. The system was never deployed, but it illustrates a fundamental problem: when historical hiring data reflects past discrimination, any model trained to predict "who was hired in the past" will learn to discriminate as the past did.

This is a case where demographic parity (or more precisely, counterfactual fairness) is the appropriate criterion. The ground truth labels (past hiring decisions) are themselves biased, so equalized odds with respect to those labels would perpetuate the bias.

Credit Scoring and Lending

The Equal Credit Opportunity Act in the United States prohibits discrimination in lending based on race, sex, religion, and other protected characteristics. Yet studies have consistently shown that algorithmic lending systems produce disparate outcomes. A 2019 study by Bartlett, Morse, Stanton, and Wallace found that fintech lenders charged Black and Hispanic borrowers 7.9 basis points higher interest rates than comparable white borrowers. A 2021 study from the Consumer Financial Protection Bureau found similar disparities in mortgage lending algorithms.

The challenge in lending is that many legitimate predictive features (zip code, education, employment history) are correlated with race due to historical segregation and structural inequality. Removing the sensitive attribute alone does not help; the bias leaks through correlated features. This motivates representation learning and counterfactual fairness approaches that attempt to remove the causal influence of the sensitive attribute rather than just the attribute itself.

Healthcare Risk Prediction

Obermeyer et al. (2019) documented one of the most striking examples of algorithmic bias in healthcare. A commercial algorithm used by hospitals to identify patients who would benefit from extra care was found to systematically favor white patients over Black patients. The bias arose because the algorithm used healthcare costs as a proxy for healthcare needs. Due to unequal access to healthcare, Black patients had historically lower healthcare costs even when they were equally or more sick, causing the algorithm to systematically underestimate their needs.

This case illustrates the danger of proxy outcomes: when the measurable outcome $Y$ is not the true quantity of interest but a proxy that is itself corrupted by the bias you are trying to address. No amount of statistical fairness optimization with respect to $Y$ will fix the problem if $Y$ itself encodes the bias.

Evaluation Frameworks and Tools

The maturation of the AI fairness field has produced several open-source toolkits that make rigorous fairness evaluation accessible to practitioners.

Fairlearn (Microsoft)

Fairlearn provides fairness assessment and mitigation algorithms integrated with the scikit-learn ecosystem. Key capabilities include:

• Assessment: MetricFrame computes any metric disaggregated by group, enabling rapid identification of disparities across all standard fairness criteria.
• Mitigation: Implements exponentiated gradient reduction (Agarwal et al., 2018), which reduces fair classification to a sequence of standard classification problems. It supports constraints for demographic parity, equalized odds, and bounded group loss. Also includes ThresholdOptimizer for post-processing.
• Dashboard: An interactive visualization for exploring fairness-accuracy tradeoffs across groups.

AI Fairness 360 (IBM)

AIF360 is a more comprehensive toolkit offering 70+ fairness metrics and 11 bias mitigation algorithms spanning pre-processing, in-processing, and post-processing. It includes implementations of:

• Reweighting and disparate impact remover (pre-processing)
• Prejudice remover and adversarial debiasing (in-processing)
• Calibrated equalized odds and reject option classification (post-processing)

Custom Metrics for LLMs

For LLM evaluation, practitioners increasingly build custom evaluation harnesses that combine:

• Automated counterfactual testing across demographic dimensions
• Toxicity detection disaggregated by demographic mentions (using Perspective API or similar)
• Stereotype detection using benchmarks like StereoSet and BBQ
• Human evaluation with diverse annotator pools

The key principle is that no single metric captures "fairness" for an LLM. Evaluation must be multi-dimensional, ongoing, and adapted to the specific deployment context.

The Fairness-Accuracy Tradeoff, and When It Is a False Tradeoff

A persistent concern in fairness research is that fairness constraints necessarily reduce accuracy. This framing is partially correct but often overstated.

When the tradeoff is real. The impossibility theorems guarantee that if base rates differ, some fairness constraints will force the classifier away from the accuracy-optimal solution. Corbett-Davies and Goel (2018) formalized this: any fairness constraint that is not satisfied by the unconstrained Bayes-optimal classifier will, by definition, reduce accuracy.

When the tradeoff is illusory. Several scenarios produce apparent tradeoffs that dissolve on closer examination:

1. Biased training data. If the training data is biased, the "accurate" model has learned the bias. Fairness constraints may reduce accuracy on the biased test set while improving accuracy on the true (unbiased) distribution. Wick et al. (2019) demonstrated this empirically across several datasets.
2. Underrepresented groups. Models often perform worse on minority groups simply due to less training data. Investing in better data collection for underrepresented groups can improve both fairness and aggregate accuracy.
3. Feature engineering. Better features can sometimes improve accuracy for disadvantaged groups without harming it for others. The tradeoff frontier is not fixed; it shifts with the quality of the underlying ML.
4. Model capacity. Underpowered models are forced into cruder tradeoffs. More expressive models can often find solutions that are closer to the Pareto frontier of fairness and accuracy.

The key insight is that the fairness-accuracy tradeoff is a property of a specific model on specific data, not a universal law. Before accepting a tradeoff, practitioners should ask whether better data, features, or models could shrink or eliminate it.

The Regulatory Landscape

The regulatory environment for AI fairness is evolving rapidly and unevenly across jurisdictions.

EU AI Act

The European Union's AI Act, which entered force in August 2024 with a phased implementation schedule, establishes the most comprehensive regulatory framework for AI fairness. High-risk AI systems (including those used in employment, credit, law enforcement, and education) must undergo conformity assessments that include bias testing and documentation. The Act mandates:

• Risk assessments that specifically address bias and discrimination
• Technical documentation of bias testing methodologies and results
• Post-market monitoring for emerging fairness issues
• Transparency obligations that include informing affected individuals about automated decision-making

United States

The US regulatory landscape remains more fragmented, and its federal trajectory reversed. The Biden administration's 2023 Executive Order on AI, which directed federal agencies to address algorithmic discrimination, was rescinded in January 2025 and replaced by an order focused on removing barriers to AI development. Earlier bias guidance from agencies like the EEOC, CFPB, and FTC still stands on the books, and the NIST AI Risk Management Framework (AI RMF) remains a voluntary standard that includes fairness considerations, but federal fairness oversight is now considerably weaker than the EU's. The action has shifted to the states, unevenly. New York City's Local Law 144 requires bias audits for automated employment decision tools. Colorado's AI Act, the first comprehensive state framework when it passed in 2024, was repeatedly delayed and then narrowed in 2026 (SB 189) to strip out its core algorithmic-discrimination duties before it ever took effect.

Emerging Global Standards

The IEEE 7010 standard for ethical AI and the ISO/IEC 24027 standard for bias in AI systems provide international frameworks. China's Algorithmic Recommendation Management Provisions and the interim measures for generative AI include anti-discrimination requirements, though enforcement mechanisms differ significantly from Western approaches.

For practitioners, the trajectory has split by jurisdiction: the EU and several other regimes are hardening fairness auditing and documentation into legal requirements, while the US has pulled back at the federal level and left a patchwork of state laws behind it. Building fairness evaluation into the ML pipeline is still forward-looking risk management, because the audit obligation you actually face depends on where your users are, not where you build.

Key Takeaways

1. Fairness is not one thing; it is a family of mathematically incompatible properties. Demographic parity, equalized odds, calibration, individual fairness, and counterfactual fairness each capture different intuitions about what "fair" means. Practitioners must choose, and the choice has consequences.

2. The impossibility theorems are real and unavoidable. When base rates differ across groups, you cannot simultaneously satisfy calibration, equalized odds, and demographic parity. This is not a limitation of current methods; it is a mathematical fact. Any system that claims to satisfy all criteria simultaneously is either operating in a trivial case or misrepresenting its properties.

3. The choice of fairness criterion is a social decision, not a technical one. The COMPAS case demonstrates this vividly. The mathematics can inform the tradeoffs, but the ultimate choice reflects values, priorities, and the specific context of deployment. This choice should be made deliberately, transparently, and with input from affected stakeholders.

4. Bias mitigation works at every stage of the pipeline. Pre-processing (data re-weighting, representation learning), in-processing (adversarial debiasing, constrained optimization), and post-processing (threshold adjustment, calibrated equalized odds) provide complementary tools. The best approach often combines interventions at multiple stages.

5. LLM fairness is a harder, less well-defined problem. The open-ended nature of language generation, the latent encoding of bias in training corpora, and the difficulty of defining fairness metrics for free-form text make LLM fairness a frontier research challenge. Current approaches (data curation, RLHF, benchmark evaluation) are necessary but insufficient.

6. The fairness-accuracy tradeoff is often smaller than feared. Biased data, underrepresented groups, and underpowered models inflate the apparent tradeoff. Better data, features, and models can shrink it. Before accepting a fairness cost, ask whether the tradeoff frontier itself can be improved.

7. Regulation is diverging, not converging. The EU AI Act and several international standards are hardening fairness auditing into a legal requirement, while the US rescinded its 2023 federal AI order in 2025 and left a patchwork of state laws, several of which (Colorado's among them) have themselves been delayed or narrowed. Where your users are, not where you build, decides the obligation you face, so building fairness evaluation into the pipeline is risk management regardless of which way your local wind is blowing.

8. Start with measurement. Before mitigating, measure. Use Fairlearn, AIF360, or custom metrics to understand your system's current fairness profile across multiple criteria. You cannot improve what you do not measure, and you cannot make principled tradeoffs without understanding the tradeoff frontier you face.

The quantization of ethics (the translation of moral intuitions into mathematical constraints) is inherently lossy. No formal criterion perfectly captures what we mean by "fair." But the alternative to imperfect formalization is no formalization at all, which means no measurement, no accountability, and no principled basis for improvement. The mathematics of fairness does not give us easy answers. It gives us something more valuable: the ability to ask precise questions and understand the exact costs of the answers we choose.

This article is part of our research series on AI Ethics. For related topics, see our coverage of Mechanistic Interpretability for understanding model internals and The Open-Source LLM Power Shift for how open-source models enable independent fairness auditing.