What it means
Human oversight is the ability of people to monitor, understand, intervene in and, if necessary, stop an AI system, both in individual cases and at the level of the whole system. The EU AI Act requires that high-risk AI systems are designed to allow effective human oversight.
Real-world example
A transport authority uses AI to prioritise road maintenance requests. Staff can see why each request received its priority, adjust priorities manually, and review weekly reports on how priorities are distributed across districts. A designated team can pause the system if it shows unexpected patterns, such as consistently ranking certain neighbourhoods lower.
Why it matters for AI adoption
Oversight requires more than a person being present. People need understandable information, training, clear responsibilities and real powers to intervene. Designing oversight is a combination of interface design, process design and governance.
Often confused with
Human-in-the-loop is one way of providing oversight, through review of individual decisions. Oversight also includes monitoring patterns, auditing and the power to stop the system.
Questions to ask
- Who is responsible for overseeing this system, and what can they see and change?
- How can the system be paused or stopped, and who decides?