How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

Summary

Cloudflare's Big Pineapple platform, which powers 1.1.1.1 and other DNS services, stores over 250 billion cache entries simultaneously. To decrease fleet-wide memory consumption, engineers implemented five changes to in-memory cache data structures. Key modifications included swapping growable types like Vec and String for Box slices to eliminate capacity overhead, combining distinct record sections into a single list indexed by offsets, omitting redundant owner domain names, and serializing record bytes into contiguous buffers. In benchmarks, these optimizations decreased per-entry memory footprints by 56% while improving cache lookup latency by 19% and insert throughput by 43%. Across production instances, the rollout reduced working-set memory by approximately 100 terabytes.

Takeaways

1. 1

   Replacing 8 Vec and String fields with Box slices removed capacity fields and excess heap allocations, saving 64 bytes per entry and over 15 terabytes across the fleet.

2. 2

   Merging answer, authority, and additional record sections into a single list with u16 offsets replaced two sets of 8-byte pointers and lengths, saving 28 bytes per entry.

3. 3

   Contiguous byte buffer layouts allowed direct copying of wire-format bytes for common types like A, AAAA, TXT, and DNSSEC records, cutting lookup latency by 5% and boosting insert throughput by 13%.