How We Scaled Our Cache and Got a Good Night's Sleep

Summary

Growing business load on the Common Data Service (CDS) created potential bottlenecks for its single-threaded Redis cache on ElastiCache, necessitating horizontal scaling for greater capacity and throughput. After ruling out master-slave replication and intermediate Twemproxy setups due to memory constraints and proxy I/O bottlenecks, the team implemented client-side sharding. Using an internal Go package for consistent hashing, CDS instances hash cache keys locally to determine the target shard. The implementation encapsulates hashing inside a thin `ShardedCache` wrapper sharing the original cache interface while supporting Ketama and custom hash functions. Deploying via double-writing cron jobs during off-peak hours reduced database read pressure and improved P99 latency.

Takeaways

1. 1

   Master-slave Redis replication failed to solve memory space limitations, while Twemproxy was rejected because it introduces an I/O bottleneck under heavy load.

2. 2

   The `ShardedCache` was implemented in Go as a thin wrapper embedding the standard `Cache` interface to enable seamless implementation swapping and custom hashing injection.

3. 3

   A two-day migration strategy utilized off-peak deployment alongside double-writing cron jobs to warm up the new shards and prevent database query spikes.