What is Human-in-the-Loop (HITL)?

Summary

Human-in-the-loop (HITL) is an AI and machine learning approach that places people in training, supervision, or decision-making to improve accuracy, safety, and ethical alignment. Its feedback loop can include data labeling, output review, escalation, approval, override, and continuous feedback, with confidence thresholds and risk scoring routing only selected decisions to people. The explainer distinguishes HITL, where review occurs before flagged actions, from human-on-the-loop monitoring and human-over-the-loop governance, and separates HITL from RLHF, a training-specific technique. It describes uses in medical imaging, moderation, autonomous vehicles, financial services, and AI agents handling consequential actions. Databricks Agent Bricks is presented as supporting governed traces and Agent Learning from Human Feedback, including a case where 32 feedback items improved instruction-following from roughly 12% to 80%.

Context

AI systems can make mistakes in unfamiliar, ambiguous, high-stakes, regulated, or consequential situations. The source presents human oversight as a way to improve reliability, reduce bias, support safety and compliance, and maintain trust while retaining automation.

Approach / What changed

HITL routes selected decisions to people through data labeling, training feedback, inference review, escalation, approval, override, and continuous feedback. Confidence thresholds and risk scoring limit review to relevant cases, while Databricks describes governed traces, Unity Catalog, and Agent Learning from Human Feedback for production AI agents.

Takeaways

• HITL is broader than RLHF: it includes training, production review, data labeling, and approval of agent actions, while RLHF specifically uses ranked or rated responses to guide model training.
• Confidence thresholds and risk scoring make HITL scalable by sending only uncertain, unusual, sensitive, or high-risk decisions to human reviewers.
• In an Agent Bricks case study, 32 pieces of human feedback improved a Q&A agent’s ability to follow expert instructions from roughly 12% to 80%.

Related reading