The Architecture of Schemaless, Uber Engineering's Trip Datastore Using MySQL

Summary

Schemaless is Uber's scalable, fault-tolerant datastore whose immutable cells are identified by row key, column name, and ref key, while JSON content remains schemaless and supports eventually consistent secondary indexes. Its worker nodes accept HTTP requests, route them to storage nodes, and aggregate results, while storage data is partitioned into typically 4096 shards, each replicated across one master and two minions in multiple data centers. Writes go to the master and replicate asynchronously, with idempotent retries, circuit breakers, failover, and configurable reads from masters or minions. To reduce loss during master failure, buffered writes persist each request to a primary and secondary cluster, though buffered data may not be immediately readable. Each shard maps to a MySQL database, where an auto-increment added_id primary key supports linear insertion and trigger processing, and MessagePack with ZLib compresses cell JSON blobs.

Takeaways

1. 1

   Each storage cluster has one master and two minions, with minions distributed across multiple data centers; reads can use any available node, while writes must initially go to the master.

2. 2

   Buffered writes send requests to a secondary cluster before the primary and require both writes to succeed, reducing the chance of data loss when asynchronous MySQL replication has not completed.

3. 3

   Each Schemaless shard is a separate MySQL database with an entity table and index tables; the auto-increment added_id primary key supports linear disk insertion and ordered trigger processing.