The Cost of Not Knowing MongoDB, Part 3: appV6R0 to appV6R4

Summary

To alleviate a disk throughput bottleneck on a MongoDB server, this optimization pass focuses on shrinking document sizes across application revisions appV6R0 through appV6R4. The implementation replaces an array of items with a dynamic schema document, using date segments such as days or combined months and days as field names to store status totals. Redundant date components already encoded in the binary _id field are omitted from internal document keys. Bulk upsert operations rely on updateOne with the $inc operator to update matching date keys or create documents when missing. Aggregation pipelines process reports using the $objectToArray and $reduce operations to calculate totals across date intervals while preserving a single index on _id.

Context

The MongoDB server faced a disk throughput bottleneck, creating the need to reduce overall document size in the application.

Approach / What changed

Developers replaced array storage with a dynamic schema document where date values serve as dynamic field names storing status totals, omitted date parts already present in the binary _id, and used updateOne with $inc alongside $objectToArray aggregation pipelines.

Takeaways

• Changing the items structure from an array to a dynamic schema document allows date values to act as field keys, reducing repetitive date storage.
• Omitting year and month components from dynamic keys is viable when those components are already encoded in the binary _id field.
• When benchmarked with 500 million events, appV6R1 lowered total data size to 8.19GB and total size per event to 20.2B compared to appV5R3.

Related reading