Loading…
Human-in-the-Loop
7 posts about Human-in-the-Loop. Every summary links to the original.
How the FDA is building a secure, AI-ready data foundation on Databricks for Government
The FDA built HALO (Harmonized AI and Lifecycle Operations for Data) as a secure, governed, AI-ready enterprise data platform for modernizing siloed systems without interrupting regulatory work. Its move to Databricks on AWS GovCloud, following FedRAMP High authorization sponsorship, added Unity Catalog as a governance layer across a multi-tenant architecture, while Terraform-based security patterns, PrivateLink, customer-managed keys, and the compliance security profile support regulated workloads. The agency migrated more than 5,000 users and 8,000 jobs and pipelines with zero downtime, refactoring over 1,000 pipelines and 4,000 notebooks. After onboarding eight centers and 30 programs, FDA reported query responses improving over 30%, compute costs falling over 20%, and provisioning and sharing time dropping over 75%. HALO also supports responsible AI use cases such as MARS, with humans retaining decision authority.
Filippo Seracini, Vijay RajaWhat is Human-in-the-Loop (HITL)?
Human-in-the-loop (HITL) is an AI and machine learning approach that places people in training, supervision, or decision-making to improve accuracy, safety, and ethical alignment. Its feedback loop can include data labeling, output review, escalation, approval, override, and continuous feedback, with confidence thresholds and risk scoring routing only selected decisions to people. The explainer distinguishes HITL, where review occurs before flagged actions, from human-on-the-loop monitoring and human-over-the-loop governance, and separates HITL from RLHF, a training-specific technique. It describes uses in medical imaging, moderation, autonomous vehicles, financial services, and AI agents handling consequential actions. Databricks Agent Bricks is presented as supporting governed traces and Agent Learning from Human Feedback, including a case where 32 feedback items improved instruction-following from roughly 12% to 80%.
Databricks StaffThe Rosetta stone of CPS: Claroty’s AI-powered library
CPS asset identification is difficult because 88% of CPS assets do not transmit an exact product code and 76% use product codes that differ from the vendor's official records. Claroty's AI-Powered CPS Library addresses this entity-resolution problem with a hybrid architecture that combines classic matching, statistical inference, generative AI, NLP, reasoning agents, and human-in-the-loop review to reconcile protocol data, vendor records, manuals, and vulnerability information. Its Databricks-based lakehouse uses Delta Lake, Unity Catalog, Spark pipelines, Model Serving, MLflow evaluations, Lakebase, and Databricks Apps to maintain governed, auditable mappings across a catalog of more than 17 million assets. The post reports 25% improved vulnerability-attribution accuracy and says early tests gave 56% of analyzed devices new or updated recommendations for outdated firmware, while low-confidence mappings remain subject to expert review.
Ben Hazan, Anton Berlinsky, Ohad Avni, Itay Wagner, Guy Zalcman, Dor Bdolach, Ravid Ariely, Gal SberroOperationalizing AI for public sector fraud prevention
Public-sector agencies face fraud methods including synthetic identities, deepfake-enhanced documentation, and personalized social engineering, while legacy controls remain fragmented and difficult to scale. The fictional Services Bureau demonstrates an operating model that combines Databricks Apps, Lakebase, Unity Catalog, Delta Sharing, Agent Bricks, and AI/BI Genie in a single fraud-operations environment. Governed data lands in Delta tables; Unity Catalog applies attribute-based access control, masks PII by role, and provides lineage, while agents connect live lakehouse queries, agency policies, and external fraud signals through MCP. Analysts review evidence and recommendations, then approve, override, or escalate cases, keeping human judgment central. The described workflow turns weeks of manual investigation into a day, supports dashboards and conversational SQL, and is presented as making fraud decisions faster, more secure, transparent, and defensible.
Johnathan Tafoya, Kacey HertanRamp ·
Closing the Books Without the Spreadsheet Shuffle: My Fall Internship at Ramp
During a fall internship at Ramp, the author worked on reducing repetitive accounting work around month-end close, focusing on amortization and ERP mapping workflows. One project uses receipt data, merchant category codes, customer history, and embedding similarity to detect likely prepaid expenses and suggest service periods and accounts or templates, with recommendations reviewable before application. A second built Ramp-Native Amortization to create schedules, generate periodic debit and credit journal entries, handle catch-ups and reclassifications, and sync entries to ERPs through a dashboard; transaction and reimbursement support was nearing alpha production release, while bill pay was next. The third generates high-confidence mapping-rule suggestions from customer coding patterns on a nightly schedule. Across the work, backtesting, accuracy metrics, and human workflow research support automation intended to earn trust rather than remove accountant judgment.
Timothy KimRamp ·
How To Build Agents Users Can Trust
Ramp describes how it built and shipped LLM-backed agents for expense management, focusing on the trust risks created by low-quality or unexpected finance outputs. It recommends choosing ambiguous, high-volume problems with asymmetric upside, then combining reasoning and citations grounded in product or user-provided context with an explicit “Needs review” path when the agent is unsure. The policy agent uses expense policies to approve, reject, or escalate expenses, while deterministic rules such as dollar limits, vendor blocklists, and category restrictions constrain autonomy. Ramp reports that more than 65% of approvals are fully handled by the agent, while users can progressively move from suggestions to autonomous actions through workflow-based controls and evaluation.
CJ Enright, Will Koh, Felix ZhangRamp ·
How Ramp builds customer-first AI
Ramp describes principles for building customer-focused AI, contrasting embedded outcomes with superficial chatbot experiences it calls “AI washing.” The approach emphasizes integrating models into workflows, separating general models trained on aggregated and masked data from sensitive models that use private customer data temporarily through in-context learning, and requiring explicit opt-in for sharing contract data. It prioritizes user control and feedback over lengthy explanations, while using guardrails that constrain outputs through predefined interactive data blocks rather than relying only on censorship filters. Ramp cites contract term extraction, spend intelligence, pricing benchmarks, Copilot, and Jsonformer as examples of these practices. The stated goal is AI that saves customers time and money while protecting privacy and producing safer, more useful results.
Yunyu Lin
