label Article

Mitigating Bias in Machine Learning Models: Where It Enters and What Works

calendar_today November 23, 2025

schedule 5-min Read

person By Dominic Fui Dodzi-Nusenu

Most bias work fails in the same way: the team measures the model, fixes what the measurement showed, and the real-world outcome does not move. Bias does not enter machine learning at one point, so it cannot be removed at one point. It enters through the data, through the target you chose to predict, and through the loop your own predictions create once the system is live — and each needs a different intervention.

Published: November 2025 · updated August 2026 · Author: Dominic Fui Dodzi-Nusenu · Reading time: ~9 minutes

Disclosure: This article was drafted using generative AI assistance and subsequently edited, verified, and expanded by human legal and tech policy specialists. The organizational case study presented below is a composite hypothetical scenario designed to illustrate compliance risks.

In brief

Bias enters at three distinct points — the data, the target, and the deployment feedback loop. Only the first is fixed by better sampling.
Deleting the protected attribute rarely removes the disparity; it reliably removes your ability to measure it.
Fairness criteria are mathematically incompatible where base rates differ. No tool can choose for you; the choice has to be recorded.
Article 10(5) permits processing special-category data to detect and correct bias, under six cumulative conditions.
Article 10 asks for an evidence trail — what you examined, found, changed and accepted — not a single score.

Where bias actually enters

Bias is usually discussed as though it were a property of the training data. It is not one thing and it does not enter at one point. Treating it as a data problem alone is why teams sample more carefully, watch the metric barely move, and conclude the problem is intractable.

The training data

The familiar case: a group is under-represented, so the model learns less about it and performs worse on it. This is the one sampling actually fixes, and it is the minority of real cases.

The label you chose to predict

The harder case, and the one that survives every data-quality fix. If you predict who was approved before rather than who repaid, the model reproduces the previous decision-maker faithfully — including whatever that decision-maker got wrong. The data is accurate; the target encodes the historical outcome rather than the thing you care about. No amount of resampling repairs a target that measures the wrong quantity.

The deployment loop

A model that declines an applicant never observes whether they would have repaid. The next training round therefore learns from a dataset the model itself censored, and the initial skew hardens. The Regulation names this directly: Article 15(4) requires high-risk systems to be designed so as to “eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops)”, and requires that any such loops be “duly addressed with appropriate mitigation measures”. That is an architectural obligation, not a data-cleaning one.

Why deleting the protected attribute does not work

The intuitive first move is to drop the column. It rarely changes the outcome, because the attribute is reconstructed from everything else: postcode, name, employment history, education, device, purchase categories. Remove one proxy and the model redistributes weight across the remaining correlated features.

Deleting the column has one reliable effect. It removes your ability to measure the disparity, while leaving the disparity in place. You have not made the system fairer; you have made it unauditable — and under Article 10 the obligation is to examine and document, so an unmeasurable system is in a worse position, not a safer one.

The three places you can intervene

Mitigation techniques divide by when they act. Each buys something different and costs something different, and the choice is an engineering decision you should be able to justify in writing.

Before training

Reweighting, resampling, or repairing the representation before the model sees it. Model-agnostic and easy to retrofit, so it is usually the first thing to try. It cannot fix a mis-specified target, and it changes the data your documentation describes — so the transformation itself becomes part of the record.

During training

Adding a fairness constraint or penalty to the objective, so the optimiser trades accuracy against disparity explicitly. This gives the most direct control and makes the trade-off visible rather than implicit. It requires access to the training pipeline, which rules it out for most bought-in models.

After training

Adjusting thresholds per group, or calibrating outputs, leaving the model untouched. Often the only option for a vendor system you cannot retrain. Note that group-specific thresholds are themselves a decision with legal weight in Germany under the AGG, and are not something to switch on quietly.

The metrics disagree, and that is arithmetic

Teams frequently discover that demographic parity improves while equalised odds degrades, and look for a bug. There is no bug. Where base rates genuinely differ between groups, the common fairness criteria are mathematically incompatible — you cannot satisfy them simultaneously except in degenerate cases. This is a proven result, not a limitation of a library.

The practical consequence is that no tool can pick the metric for you, and any vendor implying otherwise is overselling. What you can do is state which criterion you adopted, why it fits this decision, what it costs on the others, and who approved that. The Regulation does not name a metric; it requires the examination and the reasoning behind it.

You are probably allowed to collect the data you need

The most common blocker we hear is that GDPRVerordnung (EU) 2016/679 — die Datenschutz-GrundverordnungDie Datenschutz-Grundverordnung der EU. Neben Rechtsgrundlage, Zweckbindung und Betroffenenrechten knüpft Artikel 35 eine Datenschutz-Folgenabschätzung an Verarbeitungen, die voraussichtlich ein hohes Risiko zur Folge haben — die Vorschrift, der ein KI-Vorhaben am häufigsten begegnet und die sich mit einer Risikobewertung nach der KI-Verordnung überschneidet, ohne dasselbe Dokument zu sein.Gilt fürAlle Verantwortlichen und Auftragsverarbeiter, die personenbezogene Daten von Personen in der EU verarbeiten.Zum Quelltext des Rechtsakts (öffnet in einem neuen Tab)Allgemeine Informationen zum genannten Rechtsakt, keine Rechtsberatung. forbids holding the attribute, so fairness cannot be measured at all. That is not what the AI Act says, and the point is worth knowing precisely.

Article 10(5) permits providers of high-risk systems to process special categories of personal data where it is strictly necessary to detect and correct bias — subject to six cumulative conditions, all of which must hold:

other data, including synthetic or anonymised data, cannot effectively achieve the objective;
technical limits on re-use and state-of-the-art security and privacy measures are applied;
strict access controls, documentation and confidentiality obligations are in place;
the data is not transmitted to other parties;
it is deleted once the bias is corrected or the retention period ends;
records of processing document why the objective could not be met with other data.

This is a narrow, conditional permission and not a general licence — but it does mean “we are not allowed to know” is usually a governance question rather than a legal wall. The conditions are also the design specification: if you cannot meet all six, you do not have the permission.

Case scenario: the model that got fairer and the outcome that did not

A composite, illustrative example. A recruitment screening tool is measured for disparate impact and shows a clear gap. The team removes gender, retrains, and the direct disparity drops close to zero. The launch is signed off on that number.

Six months later an internal review finds the gap intact in the hiring outcome. Two things had happened. Years of continuous employment had become the dominant feature, and it encodes career breaks. And because the screening tool filtered the candidates whose later performance was observed, each retraining cycle learned from a narrower pool — the Article 15(4) feedback loop, in production.

The measurement was accurate and the fix was real; both were aimed one layer above the cause. What would have caught it is not a better metric but measuring the outcome, not just the model output, and re-measuring after deployment rather than at sign-off.

What the law asks you to keep

Article 10(2)(f) requires “examination in view of possible biases that are likely to affect the health and safety of persons, have a negative impact on fundamental rights or lead to discrimination prohibited under Union law”, and 10(2)(g) requires “appropriate measures to detect, prevent and mitigate” those biases. Article 10(3) sets the standard for the data sets themselves: relevant, sufficiently representative, and to the best extent possible free of errors and complete in view of the intended purpose.

Read together, these ask for an evidence trail rather than a score: which biases you looked for, how you tested, what you found, what you changed, what you accepted and why. A dashboard showing today’s number satisfies none of it, because it cannot show what you examined a year ago or what you decided when the metrics conflicted. Bias examination is also not a launch gate — Article 9 frames risk management as iterative across the lifecycle, which means the record is something you keep adding to.

Can you show what your bias examination concluded a year ago?

Measure fairness per model version and retain the evidence automatically.

Start your assessment →

Frequently asked questions

Is bias mitigation only required for high-risk systems?

The Article 10 data-governance duties, including the bias examination, attach to high-risk systems. Discrimination law applies regardless of risk tier — in Germany the AGG governs the decision whatever tool produced it, so a system outside Annex III is not outside the law.

Which fairness metric should we use?

The Regulation does not name one, and the common criteria cannot all be satisfied at once where base rates differ. Choose the criterion that matches the harm the decision can cause, record why, and record what it costs on the others.

Can we measure fairness without storing protected attributes?

Proxy-based estimation exists but carries its own error, and it weakens the evidence. Article 10(5) provides a conditional route to processing the real attribute for exactly this purpose, under six cumulative conditions — that is usually the more defensible path.

We license the model from a vendor. Is this still our problem?

Obligations follow the role you occupy, and a deployer who puts a system into use under their own name in specific circumstances can take on provider obligations. Practically, the deployer holds the outcome data the examination needs, so the evidence has to be produced jointly whatever the paperwork says.

How often should bias be re-measured?

Populations, and the model’s own effect on them, move. Tie re-measurement to a cadence and to triggers — retraining, a change of population, a threshold change — rather than to a launch date, and keep each result. Article 9 treats risk management as continuous.

Does a fairness dashboard satisfy Article 10?

Not on its own. A dashboard shows the current value; the obligation is to evidence the examination, the measures taken, and the reasoning. The history and the decisions are the part an auditor asks for.

arrow_back Back to Knowledge Center