Human-in-the-loop: oversight as a governance control

Human-in-the-loop (HITL) means a person is positioned to review, correct or overturn what an AI system decides. It is one of the most frequently claimed AI controls. Placing a human in the decision path is straightforward; whether the control works depends on giving that person the authority, information, time and competence to intervene.

Three configurations, often confused

• Human in the loop. A person approves each decision before it takes effect. Highest assurance, lowest throughput. Appropriate where a single wrong decision is unacceptable.
• Human on the loop. The system acts; a person monitors and can intervene. Scales far better, and depends entirely on whether intervention is genuinely possible in the time available.
• Human in command. Nobody reviews individual decisions, but a person sets the boundaries within which the system operates, decides when and how to use it, and can stop it. Often the only honest option at high volume.

The terms come from the EU High-Level Expert Group's Ethics Guidelines for Trustworthy AI (2019), and remain in common use. Arrangements described as human in the loop often operate as on the loop or in command in practice. Documenting the configuration actually in use is a basic governance step, because each carries different obligations and different residual risk. It also matters legally: a decision with only on-the-loop or in-command oversight is usually "solely automated" for GDPR and Quebec purposes.

What makes oversight real

Five conditions, and oversight fails if any one is missing:

• Authority. The reviewer can overturn the output without seeking approval from someone else. A reviewer who must escalate to disagree is not a control.
• Information. They can see why the system produced this output: the inputs, the confidence, the factors that drove it. Reviewing a bare verdict is rubber-stamping. This is where explainability stops being academic.
• Time. Enough of it to form an independent judgment. A queue calibrated to the system's throughput rather than the human's guarantees the human becomes a formality.
• Competence. Enough domain knowledge, and enough understanding of the system's limits, to know when the output is wrong. Someone who cannot evaluate the decision cannot oversee it.
• Safety to dissent. Overturning the system carries no penalty. Where overrides are measured as errors or friction, override rates fall to zero and the control stops working.

Regulators apply the same conditions. The European Data Protection Board's predecessor, the Article 29 Working Party, said in its automated decision-making guidelines that human involvement must be meaningful, carried out by someone with "the authority and competence to change the decision", and not a token gesture. California's ADMT regulations use a similar test, as does Article 26 of the EU AI Act.

Automation bias is the main failure mode

People defer to automated output, and defer more as the system becomes more reliable. As a result, the better the system performs, the less genuine scrutiny each decision receives, so accuracy erodes the control intended to catch the rare failure. Human oversight is weakest on the unusual case that a well-performing system gets wrong.

The evidence is consistent across decades and domains:

• Clinical reading. In a 2023 study in Radiology, 27 radiologists read mammograms with an AI that sometimes suggested the wrong category. When the AI was wrong, inexperienced readers' accuracy fell from about 80% to about 20%; very experienced readers fell from about 82% to about 45%. Expertise reduced the effect but did not remove it.
• Human factors research. Parasuraman and Manzey (2010) found that automation bias affects experts as well as novices and is not fixed by training alone; it is driven by workload, trust and system design.
• Public-sector policy. Ben Green's 2022 review of 41 government human-oversight policies concluded that people often cannot perform the oversight those policies assume, and that the policies can lend false legitimacy to flawed systems.
• Generative AI. The International AI Safety Report 2026 notes early evidence that reliance on AI tools can weaken critical thinking and encourage automation bias.

The measurable symptom is override rate. An oversight arrangement that is not monitored for override rate is unevidenced.

What regulation requires

EU AI Act

Human oversight is an explicit obligation for high-risk systems under the EU AI Act, and the drafting is specific. Article 14 requires providers to design high-risk systems "in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons". The people assigned to oversight must be enabled, as appropriate and proportionate:

• to "properly understand the relevant capacities and limitations" of the system and monitor its operation;
• to "remain aware of the possible tendency of automatically relying or over-relying on the output" (the Act names this automation bias);
• to "correctly interpret" the output;
• to "decide, in any particular situation, not to use" the system or to "disregard, override or reverse" its output;
• to "intervene in the operation" or "interrupt the system through a 'stop' button or a similar procedure".

For remote biometric identification, Article 14(5) goes further: no action may be taken on an identification unless it has been separately verified by at least two competent people, subject to exceptions in law enforcement, migration and border control.

Article 26 puts the matching duty on deployers: they "shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support". Deployers must also use the system according to its instructions, monitor it, suspend use and inform the provider when they see a serious risk, keep logs for at least six months, and inform workers and affected people. This is a competence and authority requirement, not a staffing one; assigning a name to the role does not by itself discharge it.

Timing (status as of October 2026). The Digital Omnibus on AI (Regulation (EU) 2026/1744) entered into force on 27 July 2026 and moved the high-risk obligations, including Articles 14 and 26, from 2 August 2026 to 2 December 2027 for Annex III systems (employment, credit, education, essential services and others) and 2 August 2028 for high-risk AI in Annex I products. The postponement extends the preparation period; the substance of the obligations is unchanged.

GDPR Article 22 and the CJEU

GDPR Article 22 gives people the right not to be subject to a decision "based solely on automated processing" that has legal or similarly significant effects, unless an exception applies (contract, law or explicit consent), and even then they can obtain human intervention, express their view and contest the decision. Two rulings shape how this works:

• SCHUFA (C-634/21, December 2023). A credit score generated automatically is itself an automated decision where a lender "draws strongly" on it to grant or refuse credit. A human signing off on a score they do not question does not take the decision outside Article 22.
• Dun & Bradstreet Austria (C-203/22, February 2025). People are entitled to an explanation of the procedure and principles actually applied to their data, in a form they can understand. Controllers cannot refuse simply by invoking trade secrets; contested information goes to the regulator or court, which balances the interests.

Together they mean the human in the loop must be real for the decision to count as human, and the explanation the human relies on must be one the affected person can also get.

Canada

• Quebec Law 25. Section 12.1 of the private-sector privacy act requires notice when a decision is made exclusively by automated processing and gives the person an opportunity to submit observations to a staff member who can review the decision.
• Federal government. The Treasury Board Directive on Automated Decision-Making requires human intervention before final decisions for higher-impact systems, scaled by an algorithmic impact assessment.
• Bill C-36. The Protecting Privacy and Consumer Data Act, tabled on 15 June 2026 and not law as of October 2026, would require explanations of automated decision systems with significant effects.

United States

• California. As of October 2026, the CCPA automated decision-making technology rules apply from 1 January 2027 to significant decisions in areas such as lending, housing, employment, education and healthcare. Technology that replaces or substantially replaces human decision-making triggers notice, opt-out and access rights. A business can avoid the opt-out by offering an appeal to a human reviewer who knows how to interpret the output and has authority to change the decision.
• Colorado. SB 26-189, signed 14 May 2026 and operative 1 January 2027, keeps a right to human review after adverse consequential decisions. It replaced SB 24-205, whose enforcement a federal court barred on 27 April 2026 in xAI v. Weiser (D. Colo.).

Frameworks

In the NIST AI RMF, oversight appears in GOVERN 3.2 (roles for human-AI configurations and oversight), MAP 3.5 (human oversight processes defined, assessed and documented) and MANAGE 2.4 (mechanisms to supersede, disengage or deactivate a system). ISO/IEC 42001 reaches it through the AI system impact assessment, which asks what the system could do to people. That question usually determines how much human involvement is proportionate.

Human oversight of AI agents

Agentic AI breaks the per-decision model. An agent that plans, calls tools and takes dozens of actions to finish a task does not produce a single output for a human to approve. The International AI Safety Report 2026 flags agents as a heightened risk because they act autonomously, "making it harder for humans to intervene before failures cause harm".

Oversight for agents shifts from reviewing outputs to designing boundaries:

• Approval gates for consequential actions. Payments, deletions, external communications and changes to records wait for a human. Everything else runs.
• Least privilege. The agent gets only the tools, data and permissions the task needs, with short-lived credentials.
• Interruptibility. A working stop mechanism, the agentic version of Article 14's "stop button", that halts the agent in a safe state.
• Legible action logs. A trace a reviewer can read after the fact, showing what the agent did and why.
• Monitoring by exception. Automated monitors, including other models, flag anomalous actions for human review, because no person can watch every step.

Singapore's Model AI Governance Framework for Agentic AI (2026) and NIST's AI Agent Standards Initiative (2026) are early references as of October 2026.

Evidence of human oversight

Auditors and regulators expect evidence, not a policy statement. Useful evidence includes:

• A configuration record for each system: in, on or in command, and why that level matches the consequence of failure.
• Role definitions and training records showing reviewers have the competence and authority the law requires.
• Override and escalation rates by reviewer and over time, with investigation when rates approach zero.
• Review time per decision compared with what a genuine review needs.
• Quality sampling in which a second reviewer re-examines a sample of approved decisions.
• Stop-mechanism tests showing the system can be halted, and that someone has done it in an exercise.

These belong in the AI audit evidence file and in board reporting. See AI oversight at board level.

Where human oversight does not help

Human oversight has known limits, and stating them keeps the control credible.

• At volume and speed. Real-time decisions at machine scale cannot be individually reviewed. Human-in-command with hard boundaries is the honest design; claiming per-decision review is not.
• Against deception. Research on models that recognize evaluation and behave differently, or that hide actions in their stated reasoning, shows oversight can be defeated by a system that knows it is being watched. See AI safety.
• Against attacks the reviewer cannot see. An indirect prompt injection that reaches a model through a retrieved document is invisible in the output a human reviews.
• When the human introduces the bias. Oversight adds a second judgment, which is not automatically a better one. Reviewers carry their own biases, and unmonitored override patterns can encode them.
• When the explanation is wrong. A plausible but unfaithful explanation can make a reviewer more confident in a bad output, not less.

These limits are not an argument against human oversight, but against treating it as a universal answer.

Related references

For the discipline this control sits inside, see AI governance and AI accountability. For the explanations reviewers depend on, see explainable AI. For what the board should see about it, see AI oversight at board level. For the harms oversight is meant to catch, see AI risks.

Sources and references

• Regulation (EU) 2024/1689 (AI Act), Articles 14 and 26: https://eur-lex.europa.eu/eli/reg/2024/1689/oj
• Regulation (EU) 2026/1744 (Digital Omnibus on AI): https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L_202601744
• Regulation (EU) 2016/679 (GDPR), Article 22: https://eur-lex.europa.eu/eli/reg/2016/679/oj
• CJEU, SCHUFA Holding (Scoring), C-634/21 (7 December 2023): https://curia.europa.eu/juris/liste.jsf?num=C-634/21
• CJEU, Dun & Bradstreet Austria, C-203/22 (27 February 2025): https://curia.europa.eu/juris/liste.jsf?num=C-203/22
• Article 29 Working Party, Guidelines on Automated individual decision-making and Profiling (WP251rev.01): https://ec.europa.eu/newsroom/article29/items/612053
• High-Level Expert Group on AI, Ethics Guidelines for Trustworthy AI (2019): https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
• Quebec, Act respecting the protection of personal information in the private sector (P-39.1): https://www.legisquebec.gouv.qc.ca/en/document/cs/P-39.1
• Treasury Board of Canada Secretariat, Directive on Automated Decision-Making: https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592
• Parliament of Canada, Bill C-36: https://www.parl.ca/legisinfo/en/bill/45-1/c-36
• California Privacy Protection Agency, regulations: https://cppa.ca.gov/regulations/
• Colorado SB 26-189: https://leg.colorado.gov/bill_files/116236/download
• NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
• International AI Safety Report 2026: https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026
• RSNA, "AI bias may impair accuracy of radiologists" (Dratsch et al., Radiology, 2023): https://www.rsna.org/news/2023/may/ai-bias-may-impair-accuracy
• Parasuraman and Manzey, "Complacency and Bias in Human Use of Automation" (Human Factors, 2010): https://doi.org/10.1177/0018720810376055
• Green, "The flaws of policies requiring human oversight of government algorithms" (Computer Law & Security Review, 2022): https://arxiv.org/abs/2109.05067
• IMDA, Model AI Governance Framework for Agentic AI: https://www.imda.gov.sg/-/media/imda/files/about/emerging-tech-and-research/artificial-intelligence/mgf-for-agentic-ai.pdf
• NIST, AI Agent Standards Initiative: https://www.nist.gov/artificial-intelligence/ai-agent-standards-initiative