Scaling Airbnb’s identity graph with a unified knowledge graph infrastructure

Summary

Airbnb migrated its massive identity graph from a third-party graph database vendor to a unified, in-house knowledge graph platform to solve long-tail latency and scaling bottlenecks. Built on JanusGraph with AWS DynamoDB for persistence and OpenSearch for indexing, the new architecture decouples storage operations from graph traversal logic. The migration improved P99 query latency, eliminated routine instance reboots, and supported a tenfold increase in write throughput.

Takeaways

• Airbnb built an internal graph platform using JanusGraph decoupled from storage, leveraging AWS DynamoDB for persistence and OpenSearch for indexing.
• Engineers resolved Gremlin query bottlenecks in JanusGraph by eliminating unbatched Path steps, parallelizing slice reads, and replacing heavy transactions with DynamoDB conditional writes.
• The internal platform enabled shadow-traffic benchmarking against the legacy vendor, achieving a 10x increase in write QPS while reducing long-tail query latencies.

Related reading

• Airbnb · Jun 9, 2026

  Scaling beyond one: How Airbnb evolved its data architecture for a multi-product world

  To support its expansion from Homes into Experiences and Services, Airbnb evolved its offline data warehouse architecture to balance domain-specific needs with organizational consistency. Rather than enforcing a single global pattern, data teams adopted a framework of foundational principles that paired separate data models for unique product features with monolithic models for cross-cutting domains like payments and messaging.

  Patrick Lam

• Airbnb · Jul 28, 2026

  Eval-driven development: Lessons from evaluating GenAI at scale

  Airbnb outlines its framework for eval-driven development (EDD), treating GenAI evaluation as a continuous engineering discipline rather than an afterthought. The strategy combines programmatic checks, calibrated LLM-as-a-judge evaluators, and human review to detect subtle quality regressions. By inspecting intermediate agentic execution traces and real-world failure modes, teams ensure generative models remain reliable and aligned with product requirements.

  Rohit Girme

• Airbnb · Aug 12, 2026

  Flexible Authentication: Reimagining authentication for millions of users at Airbnb

  Airbnb overhauled its login and registration architecture by transitioning to a server-driven framework called Flexible Authentication. The new system separates user identification from verification, using a backend policy engine to dynamically serve the optimal auth challenge and fallback options based on regional and user context. This shift eliminated dead ends, reduced client bundle sizes, and dramatically sped up experimentation velocity across Web, iOS, and Android.

  Jose Santos

• Airbnb · Aug 25, 2026

  Project Lighthouse — Part 3: Introducing project-lighthouse-anonymize

  Airbnb has open-sourced project-lighthouse-anonymize, a Python library powering their privacy-preserving anonymization used to measure user experience disparities. The release is accompanied by technical papers detailing Core Mondrian, an extended partition-based anonymization algorithm, alongside a framework for measuring anonymized data quality. The library enforces k-anonymity and p-sensitive k-anonymity while offering built-in data quality metrics.

  Adam Bloomston