What it means
In AI, bias means systematic patterns in a system's outputs that unfairly favour or disadvantage certain people or groups, or that consistently distort results. It can come from training data, design choices, how a problem is defined or how a system is used.
Real-world example
A recruitment team tests an AI tool that ranks applications. Testing with comparable CVs shows that candidates with career breaks, often parents and carers, are consistently ranked lower, because the historical hiring data favoured continuous employment. The team adjusts the criteria and adds regular fairness checks before using the tool.
Why it matters for AI adoption
Biased AI can lead to discrimination, unfair decisions and loss of trust, and in many contexts it can also breach equality and data protection law. Bias cannot be removed once and forgotten; it needs to be tested for with real data, monitored over time and considered in how the system is used.
Questions to ask
- How has the system been tested for unfair differences between groups of people?
- Who is responsible for monitoring bias after launch, and what happens if it is found?