---
title: "Beyond the Model: Engineering AI Infra with Scientific Judgement"
description: "Airbnb describes Insight Miner, an agent harness designed to make unstructured data investigations scalable, reproducible, and audit-ready. Its problem was a bespoke process spanning data discovery, expert review, representative datasets, and manually curated methods that could not keep pace with new languages, geographies, and LLM-based products. The shared package combines extraction, embedding, and clustering with prompt tuning, hard-example mining, contrastive labeling, and a path from unsupervised exploration to classification, while an agent handles research questions, execution, and methodological guidance. Airbnb says investigations that once took months can now take days, allowing mechanical work to scale while data scientists spend more time inspecting ambiguous evidence, testing hypotheses, and improving techniques. The tool also lets nontechnical experts run analyses across use cases including survey coding, model evaluation, and fraud patterns."
---

# Beyond the Model: Engineering AI Infra with Scientific Judgement

[Airbnb](https://yomu.fyi/company/airbnb) · AirbnbEng · Sep 15, 2026

**Type:** Explainer

## Summary

Airbnb describes Insight Miner, an agent harness designed to make unstructured data investigations scalable, reproducible, and audit-ready. Its problem was a bespoke process spanning data discovery, expert review, representative datasets, and manually curated methods that could not keep pace with new languages, geographies, and LLM-based products. The shared package combines extraction, embedding, and clustering with prompt tuning, hard-example mining, contrastive labeling, and a path from unsupervised exploration to classification, while an agent handles research questions, execution, and methodological guidance. Airbnb says investigations that once took months can now take days, allowing mechanical work to scale while data scientists spend more time inspecting ambiguous evidence, testing hypotheses, and improving techniques. The tool also lets nontechnical experts run analyses across use cases including survey coding, model evaluation, and fraud patterns.

## Context

Airbnb needed to investigate the situations its planned AI customer service assistant would encounter, including rare and risky events, while expanding analysis across languages, geographies, and LLM-based products.

## Approach / What changed

Insight Miner packages extraction, embedding, clustering, prompt tuning, hard-example mining, contrastive labeling, and the progression from unsupervised exploration to classification. An agent begins from a research question in chat and serves as research partner, executor, and methods expert, while shared infrastructure scales and parallelizes mechanical work.

## Takeaways

- Insight Miner turns methods previously spread across notebooks, tables, documents, and individual judgment into a shared package that can be updated when investigations produce better techniques.
- The harness reduced investigations that previously took months to a matter of days while preserving human attention for ambiguous or strategic data, hypothesis testing, and qualitative understanding.
- Dozens of teams use Insight Miner for hundreds of investigation types; nontechnical subject-matter experts use it for survey coding, model evaluation, and fraud-pattern analysis.

## FAQ

### Which techniques does Insight Miner bring into one investigation workflow?

It combines extract, embed, and cluster with prompt tuning, hard-example mining, contrastive labeling, and a progression from unsupervised exploration to classification. These methods were previously maintained in separate notebooks and shared by copy and paste.

### Does the harness replace human judgment in data investigations?

No. It scales and parallelizes mechanical work so data scientists can spend more time inspecting ambiguous or strategic data, testing hypotheses, and building qualitative and quantitative understanding.

### Who can use Insight Miner?

It was initially designed for data science teams but expanded to dozens of teams, with particularly strong use among operations and product-insights groups. A UI also enables subject-matter experts who have never written code to conduct scaled analyses.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents), [Data Science](https://yomu.fyi/topic/data-science), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning)

- Source: [Airbnb](https://medium.com/airbnb-engineering/beyond-the-model-engineering-ai-infra-with-scientific-judgement-371316d43261)
- Source URL: https://medium.com/airbnb-engineering/beyond-the-model-engineering-ai-infra-with-scientific-judgement-371316d43261
- Ingested by Yomu: 2026-09-15T18:14:09.813Z

[Read original post](https://medium.com/airbnb-engineering/beyond-the-model-engineering-ai-infra-with-scientific-judgement-371316d43261)