Loading post…
MongoDBExplainer
The Ground Beneath the Database
Summary
On September 29th, MongoDB announced the public preview of Atlas Infinite, a new deployment option built on disaggregated storage where compute and storage run on separate machines. The shared storage layer consists of three services: a Log Service for write consensus, a Page Service keeping materialized pages warm, and an Object Index Service managing object storage. This architecture was necessary because WiredTiger’s speculative eviction and deferred writes made conventional approaches impossible. Data is encrypted on the compute node before reaching the storage layer, preventing shared services from reading documents. Benefits include unlimited storage scaling, read replicas without disk copying, and constant-time snapshots.
Context
MongoDB Atlas Infinite is a new deployment option that disaggregates compute and storage to solve the problem of storage gravity in traditional coupled architectures, where scaling, replication, and backups require copying large amounts of data.
Approach / What changed
Atlas Infinite uses a shared storage layer with three services: Log Service for write consensus, Page Service for materialized pages, and Object Index Service for object storage durability. Compute nodes are stateless and encrypt data before sending to storage, preserving WiredTiger’s deferred write efficiency while keeping the storage layer unable to read customer data.
Takeaways
Atlas Infinite disaggregates compute and storage, making compute nodes stateless and storage a shared layer with Log, Page, and Object Index services.
The architecture preserves WiredTiger’s deferred write efficiency while keeping the storage layer unable to read customer data through encryption on the compute node.
Benefits include virtually unlimited storage scaling, read replicas without disk copying, and constant-time snapshots and restores.
FAQ
How does Atlas Infinite handle WiredTiger's non-deterministic page layouts?
MongoDB built a custom solution that keeps WiredTiger's deferred writes intact and uses a Log Service for consensus, Page Service for materialized pages, and Object Index Service for durability, without requiring a canonical page image.
Can the shared storage layer read customer data?
No, data is encrypted on the compute node at the database level before being sent to the storage layer, so the shared services cannot read documents, collection names, or indexes.
What are the performance benefits of disaggregation?
Storage scaling becomes virtually unlimited, read replicas come up without copying a disk, and snapshots and restores become constant-time operations.
Source and freshness
- Source
- mongodb.com
- Published
- Added to Yomu
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.