AI Bias in Financial Services: How to Detect It and What the Law Requires
Bias in a credit model is not primarily a statistics problem. It is an evidence problem. Article 10 of the EU AI ActRèglement (UE) 2024/1689Le règlement de l'Union européenne sur l'intelligence artificielle. Il répartit les systèmes d'IA en niveaux de risque — interdit, haut risque, risque limité et risque minimal — et attache des obligations différentes au fournisseur qui construit un système et au déployeur qui l'utilise. Il est entré en vigueur le 1er août 2024 et s'applique par étapes ; le Digital Omnibus, règlement (UE) 2026/1744, en vigueur depuis le 27 juillet 2026, a reporté les étapes relatives au haut risque sans modifier les sanctions. L'article 99 les maintient à un maximum de 35 millions d'euros ou 7 % du chiffre d'affaires annuel mondial pour les pratiques interdites, et à un maximum de 15 millions d'euros ou 3 % pour la plupart des autres manquements, y compris les obligations en matière de haut risque et de transparence.S'applique àFournisseurs et déployeurs de systèmes d'IA mis sur le marché de l'UE ou utilisés dans l'UE.Prochaine échéance2 décembre 2026 — Marquage des contenus générés par l'IA au titre de l'article 50, paragraphe 2, pour les systèmes déjà sur le marché au 2 août 2026Lire le texte source (s'ouvre dans un nouvel onglet)Information générale sur l'instrument cité, ne constituant pas un conseil juridique. requires training, validation and test data to be examined for possible biases — and the obligation is to examine and record, not merely to achieve a number. Teams that optimise a fairness metric without documenting the examination have done the hard part and failed the requirement.
Published: March 2026 · updated August 2026 · Author: Dominic Fui Dodzi-Nusenu · Reading time: ~8 minutes
Disclosure: This article was drafted using generative AI assistance and subsequently edited, verified, and expanded by human legal and tech policy specialists. The organizational case study presented below is a composite hypothetical scenario designed to illustrate compliance risks.
In brief
Bias enters financial AI through three routes: historical outcomes encoded in training data, proxy variables that reconstruct a protected attribute from apparently neutral features, and measurement bias where the label itself is unequal across groups. Article 10 requires examination of data for bias and Article 15 requires declared accuracy and robustness, both from 2 December 2027 for high-risk systems; the German AGG and the Equal Treatment framework apply already. The technical trap is that the common fairness metrics — demographic parity, equalised odds, and calibration — are mathematically incompatible except in degenerate cases, so you cannot satisfy all of them and must instead choose, justify and document your choice. That justification, dated and attributable, is the artefact a supervisor actually asks for.
How bias enters a financial model
Historical outcome bias
A model trained on past lending decisions learns the past, including its distortions. If a group was historically underserved, repayment data for that group is sparse and selected — you only observe outcomes for applicants who were approved. The model then encodes the approval policy rather than creditworthiness, and does so with more consistency than the humans it replaced.
Proxy variables
Removing a protected attribute does not remove its influence. Postcode, employer, device type, transaction merchants and even application timing can jointly reconstruct ethnicity, age or gender to a high degree. This is why « we do not collect that field » is not a defence — it is often the reason the proxy went undetected.
Measurement bias
Sometimes the label is unequal. If « default » is recorded through a collections process that escalates differently across customer segments, the target variable itself carries the bias, and no amount of feature work in the model fixes a mismeasured outcome.
The metrics, and why they disagree
| Metric | What it equalises | When it is the right choice |
|---|---|---|
| Demographic parity | Approval rate across groups | Where equal access to the product is the policy goal |
| Equalised odds | True and false positive rates across groups | Where errors carry asymmetric harm and base rates genuinely differ |
| Calibration by group | Meaning of a given score across groups | Where the score is consumed downstream and must mean one thing |
| Counterfactual fairness | Outcome under a changed protected attribute | Useful diagnostically; hard to evidence at portfolio scale |
These cannot generally all hold at once. Where base rates differ between groups, calibration and equalised odds are provably incompatible, and demographic parity conflicts with both. This is a mathematical result, not a tooling limitation — so a project plan that says « achieve fairness » is unimplementable as written.
What is implementable: choose the metric that matches the harm you are trying to prevent, measure it, record the trade-off you accepted against the others, and say who decided. That record is the compliance artefact.
Case scenario: a proxy nobody looked for
A consumer lender removed age and gender from its feature set and considered the fairness question closed. A later review measured approval rates by age band and found a sharp step at the boundary between two bands. The driver was device and browser metadata, which correlated strongly with age and had been added to improve fraud detection — a different objective entirely, in a different team, with no fairness review because the feature was not about credit.
The instructive part is not the proxy. It is that the feature was introduced by a team with no reason to run a bias check, which is how most proxies arrive. Examination has to be a property of the pipeline, not of the modelling team’s diligence.
What Article 10 actually asks for
- Examine training, validation and test data for possible biases that could affect health, safety or fundamental rights, or lead to prohibited discrimination.
- Detect, prevent and mitigate those biases, with appropriate measures — the standard is appropriateness, not perfection.
- Address gaps and shortcomings in the data, and record what they are where they cannot be closed.
- Document the data’s provenance, collection, and the assumptions it encodes.
Read that list again with the reconstruction test in mind: could you show what your bias examination concluded for the model version that ran eighteen months ago? For most teams the answer today is no, and closing that is a logging and retention change rather than a modelling change.
A practical measurement routine
- Define the groups you will measure against, and record why those and not others.
- Measure at least approval rate, true positive rate and false positive rate per group, per model version — not just at launch.
- Re-measure on a fixed cadence and on every retrain, and retain the result with the version.
- Test for proxies explicitly by attempting to predict the protected attribute from your feature set; a high score there is the finding.
- Record the metric chosen, the trade-off accepted, the person accountable and the date.
Can you show what your bias examination concluded a year ago?
Measure fairness per model version and retain the evidence automatically.
Related reading
- EU AI Act Compliance Checklist for High-Risk AI Systems
- EU AI Act for Financial Institutions: The 2027 Deadline
- How German Banks Are Adopting AI: Market Landscape 2026
Frequently asked questions
Can we just remove protected attributes from the model?
No. Proxy variables routinely reconstruct protected attributes from neutral features such as postcode, employer or device metadata. Removing the field often makes the effect harder to detect rather than absent, and Article 10 asks you to examine for bias, which requires measuring against the attribute you removed.
Which fairness metric does the EU AI Act require?
It does not name one. Article 10 requires examination for bias and appropriate mitigation; the choice of metric is yours, and because the common metrics are mutually incompatible, that choice must be made deliberately and documented with its rationale.
Does the AGG still apply alongside the AI Act?
Yes. German equal-treatment law applies to discriminatory outcomes today, independently of the AI Act timeline, and a fairness failure can be actionable under it well before high-risk obligations bite in December 2027.
How often should bias be re-measured?
On every retrain and on a fixed calendar cadence between retrains. Data drifts even when the model does not, so a launch-time measurement says nothing about current behaviour — and Article 72 post-market monitoring expects the ongoing view.
Is a fairness dashboard enough evidence?
Only if it retains history per model version. A dashboard showing current state cannot answer what the model did last year, which is the question an audit asks.
What if mitigating bias reduces model accuracy?
That trade-off is expected and is not itself a compliance failure. What matters is that the trade-off was identified, quantified, decided by someone with the authority to decide it, and recorded — which is exactly what Article 9 risk management is for.