Simplifying Data and Product Integrations with a Data Abstraction Layer

Summary

Uber built a domain-agnostic Data Abstraction Layer (DAL) to decouple data consumers from changing physical datasets and simplify advertiser reporting. The DAL is an RPC service that accepts a logical table, schema, time range, and filters; logical tables define the interface, while configuration describes locations, relationships, availability windows, and rollup transforms. It generates database-specific queries, runs them in parallel across systems such as an internal OLAP database, Docstore, and Apache Hive, and assembles responses with aggregation and rollup behavior determined by the request. Removing a dimension can produce an account-level response with metrics automatically rolled up, while mixed realtime and daily tables can cover different portions of a requested time range. For advertiser reporting, the approach reduced new report delivery from several weeks or months to under two days and enabled dynamic filters, report builders, and third-party API integrations.

Takeaways

1. 1

   Logical tables expose a stable data interface without specifying where or how data is retrieved; the DAL resolves the physical tables and columns needed for each request.

2. 2

   Table resolution filters candidates by requested schema and availability, then maps metric time ranges to candidates, preferring overlapping candidates closer in cardinality to the request.

3. 3

   New advertiser reports that previously took several weeks to a couple of months can now be delivered in under two days, enabling dynamic report builders and third-party API integrations.