The Cost of Not Knowing MongoDB, Part 3: appV6R0 to appV6R4

Summary

To alleviate a disk throughput bottleneck on a MongoDB server, this optimization pass focuses on shrinking document sizes across application revisions appV6R0 through appV6R4. The implementation replaces an array of items with a dynamic schema document, using date segments such as days or combined months and days as field names to store status totals. Redundant date components already encoded in the binary _id field are omitted from internal document keys. Bulk upsert operations rely on updateOne with the $inc operator to update matching date keys or create documents when missing. Aggregation pipelines process reports using the $objectToArray and $reduce operations to calculate totals across date intervals while preserving a single index on _id.

Takeaways

1. 1

   Changing the items structure from an array to a dynamic schema document allows date values to act as field keys, reducing repetitive date storage.

2. 2

   Omitting year and month components from dynamic keys is viable when those components are already encoded in the binary _id field.

3. 3

   When benchmarked with 500 million events, appV6R1 lowered total data size to 8.19GB and total size per event to 20.2B compared to appV5R3.