Shopify2023-10-18Explainer
Five Common Data Stores and When to Use Them
Summary
The article compares five common data stores—relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues—and connects each to appropriate design needs. Relational databases use schema-bound tables, primary and foreign keys, SQL, and ACID transactions for durable business-critical data, while NoSQL databases trade strict schemas for flexibility with document and wide-column models. In-memory stores such as Redis and Memcached provide fast access and configurable eviction for caching; Redis additionally supports richer data structures and disk persistence. Full-text search engines target word and substring searches, whereas Kafka uses a distributed append-only log and producer-consumer model to move data reliably between services. The article concludes that the right choice depends on whether data must be permanent, flexible, quickly accessed, searched, or transferred.
Context
Choosing a data store is a technical-design decision shaped by whether data has a fixed or flexible schema, whether it must persist permanently, and whether it needs to be temporarily stored, searched, or transferred.
Approach / What changed
The article compares relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues by describing their data models, durability, performance, persistence, scalability, and suitable use cases, with examples including MySQL, Bigtable, Redis, Memcached, Elasticsearch, and Kafka.
Takeaways
Relational databases enforce schemas and support primary keys, foreign keys, SQL, and ACID transactions, making them suited to durable, business-critical information.
Redis and Memcached are in-memory key-value stores with configurable eviction policies; Redis also supports native data structures and optional disk persistence, while Memcached is limited to caching.
Kafka is a distributed append-only log with producer-consumer access, making it useful for reliably moving data between services and scaling through distributed partitions.
Source and freshness
- Source
- shopify.engineering
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.