---
title: "Caching"
description: "95 posts about Caching, summarised, each linking to the original."
---

# Caching
> 95 posts about Caching, summarised, each linking to the original.

## Articles

### [Turboku - The Art Of Faster Heroku Apps](https://yomu.fyi/post/turboku-the-art-of-faster-heroku-apps.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Feb 5, 2020
- Type: Benchmark

Turboku is a migration path that automatically turns Heroku web dynos into Fly applications, addressing how existing Heroku apps can move to Fly’s globally distributed edge network. Fly routes requests to the closest datacenter through one global IP address, or two with IPv6, while TLS terminates closer to users and modern ECDSA certificates are automatic for Fly domains. A benchmark around the world used an existing application to compare Heroku and Fly connection stages, full requests, and cached requests. Fly reduced DNS, TCP, TLS, network, and full-request times, with the largest reported improvement appearing when caching was used: 180.16 ms versus Heroku’s 817.75 ms, a 77.91% reduction; uncached full requests fell 35.39%, while TTFB was 1.24% higher.


### [Journey to a Faster Everyday Superapp Where Every Millisecond Counts](https://yomu.fyi/post/journey-to-a-faster-everyday-superapp-where-every-millisecond-counts.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Renu Yadav
- Published: Dec 26, 2019
- Type: Problem & solution

Grab undertook an initiative to reduce startup time and improve time to interactive (TTI) on its passenger mobile app. Because local benchmarks failed to simulate real device and network conditions, the team instrumented code in production across 8–9 million daily users to capture p50 and p95 metrics. Initial gains came from caching service tiles between sessions and removing a startup animation, saving four seconds. Architectural changes followed, including converting iOS dynamic frameworks to static linking and merging others, while Android initialisation was refactored with Kotlin coroutines. Replacing a heavy third-party analytics library with an internal experimentation platform yielded further startup reductions.


### [How to Write Fast Code in Ruby on Rails](https://yomu.fyi/post/how-to-write-fast-code-in-ruby-on-rails.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 8, 2019
- Type: Tutorial

Shopify’s guide presents performance advice for Ruby on Rails across Active Record, Rails, and Ruby, while treating speed as a feature rather than the first optimization priority. It recommends understanding Active Record’s lazy query execution, selecting fewer columns, avoiding unindexed queries, using safe indexing approaches for large tables, and treating query cache as short-lived rather than dependable. For Rails applications, it covers caching, throttling expensive or abusive operations, moving long-running work into Active Job-backed queues, and reducing dependency growth to limit boot time and memory use. Ruby-specific guidance includes limiting metaprogramming and indirection, choosing O(1) hash lookups over O(n) array searches when appropriate, and reducing allocations while avoiding harmful global mutation; benchmark figures show method-definition and invocation costs can differ.


### [Optimizing Database Performance in Rails](https://yomu.fyi/post/optimizing-database-performance-in-rails.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Jonan Scheffler
- Published: Apr 15, 2019
- Type: Tutorial

Production database performance in Rails can degrade as application and data volumes grow, with slow queries, unnecessary queries, and other database bottlenecks becoming significant constraints. The post recommends establishing monitoring early, using Heroku Postgres events, data.heroku.com diagnostics, and the pg-extras pg:outliers command to locate expensive queries, then checking Rails access patterns such as unnecessary association loads. It shows how ActiveRecord includes can reduce a collection example from 11 queries to two, and explains when indexes may help, including Rails migrations for indexed columns and foreign keys, while noting storage and write costs. It also presents fragment caching with Redis as a further optimization and points to connection leaks, locks, and VACUUM as additional areas to investigate, urging benchmarking and metrics for every change.


### [Designing Resilient Systems Beyond Retries (Part 2): Bulkheading, Load Balancing, and Fallbacks](https://yomu.fyi/post/designing-resilient-systems-beyond-retries-part-2-bulkheading-load-bal.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Michael Cartmell
- Published: Mar 25, 2019
- Type: Explainer

Software systems require mechanisms beyond retries to maintain resilience during downstream outages and high traffic. Bulkheading isolates failures across infrastructure, processes, thread pools, and connection limits, preventing a single failing component from degrading an entire system. Load balancing distributes traffic across backend pools via proxies, client-side libraries, lookaside services, or sidecars, often pairing with health checks to eliminate single points of failure. When operations fail unrecoverably, fallback strategies like silent failures, local defaults, stale cache reads, and dedicated backup services enable graceful degradation. Organizations like Grab implement these approaches using internal client-side load balancers backed by etcd, cache fallbacks in microservice frameworks, and redundant core backup services.


### [Recipe for Building a Widget: How We Helped to “Peak-Shift” Demand by Helping Passengers Understand Travel Trends](https://yomu.fyi/post/recipe-for-building-a-widget-how-we-helped-to-peak-shift-demand-by-hel.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Lara PuReum Yim
- Published: Mar 7, 2019
- Type: Problem & solution

Transport demand spikes during regular commuting hours often outpace driver availability, resulting in passenger wait times and fare surges. To mitigate these imbalances, Grab created the Travel Trends Widget for its mobile feed to redistribute ride requests toward off-peak windows. The widget uses machine learning forecasting to present historical supply-demand patterns alongside pricing trends for the upcoming two hours. To handle anticipated high query rates across millions of database entries, engineers periodically load precomputed trend data into an in-memory data structure rather than querying the database per request. The feature rolled out to feeds in Singapore and Jakarta within four weeks of initial development.


### [Debugging in Ruby—Busting a Year-old Bug in Sprockets](https://yomu.fyi/post/debugging-in-ruby-busting-a-year-old-bug-in-sprockets.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Richard Schneeman
- Published: Feb 27, 2019
- Type: Tutorial

Sprockets, the Ruby asset packaging system used in Rails’ asset-processing pipeline, had a bug that made assets:precompile fail after a project directory was renamed between runs. The debugging walkthrough reproduces the failure with CodeTriage by clearing caches, compiling assets, moving the project, and compiling again, yielding an error for an outdated absolute path. It then uses bundle open, source inspection, recursive grep, and IRB to examine Sprockets’ dependency cache; the IRB cache lookup does not work because the cache key differs from the cache ID, but the on-disk record exposes an absolute asset path where a relative path is expected. The proposed fix updates loader.rb’s to\_load and to\_link handling with compress\_from\_root, and the pull request adds a regression test.


### [The Unreasonable Effectiveness of Test Retries: An Android Monorepo Case Study](https://yomu.fyi/post/the-unreasonable-effectiveness-of-test-retries-an-android-monorepo-cas.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: Alejandro Rodriguez Salamanca
- Published: Jan 8, 2019
- Type: Problem & solution

An Android monorepo expansion and larger team increased the number of tests run on master, exposing how flaky tests and infrastructure failures could erode CI confidence. Under an independence assumption, tests passing 99.95% each produced a theoretical full-suite pass rate of only 35% across 20 pipelines, while retrying each failed test once raised it to 99.95%. Shopify therefore added retries for setup failures, distinguishing failures retriable within a job from fatal failures requiring the test environment to reload, and retried failed tests up to three times. The system also flagged tests that passed after retry as warnings, since retries can hide reliability problems and slow builds. In practice, Android app pipeline pass rate rose from 31% to almost 90% after retry mechanisms and Gradle caching, while iOS rose from 67% to 97%.


### [Cache Invalidation Complexity: Rails 5.2 and Dalli Cache Store](https://yomu.fyi/post/cache-invalidation-complexity-rails-5-2-and-dalli-cache-store.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Richard Schneeman
- Published: Oct 16, 2018
- Type: Problem & solution

Rails 5.2+ applications can fail to invalidate ActiveRecord-backed cache entries when cache versioning is enabled alongside a third-party store such as dalli\_store 2.7.8 or earlier. Rails’ recyclable cache keys keep the cache\_key stable and store an ActiveRecord cache\_version in the entry, preventing unrelated entries from being evicted, but Dalli only understands the key and therefore repeatedly returns stale data. The post explains how to check the production cache\_store and active\_record.cache\_versioning settings, then compares switching to Rails’ :mem\_cache\_store, disabling versioning, and upgrading Dalli to 2.7.9+. Its recommendation is to use :mem\_cache\_store and flush the cache, while noting that Dalli 2.7.9 fixes invalidation by putting the version into the key rather than implementing recyclable keys.


### [Heroku CLI: Completing Autocomplete](https://yomu.fyi/post/heroku-cli-completing-autocomplete.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Philipe Navarro
- Published: Jun 12, 2018
- Type: Explainer

Heroku CLI Autocomplete had to support a dynamic, plugin-based command set while completing values that often reside behind the Heroku API. Instead of static shell case statements, its cache builder iterates through installed commands and generates setters that supply command, flag, and value completions when the user presses Tab. A shim sourced from the user’s .bashrc or .zshrc limits profile changes, fails silently when unavailable, and permits updates without repeated edits. API-backed values are cached on disk, while oclif custom hooks invalidate or rebuild relevant caches after commands change data. The completion function also parses the partially entered command line to provide contextual results, such as add-on aliases for a specified app; the work is being incorporated into an oclif plugin.


### [Dealing with the Meltdown Patch at Grab](https://yomu.fyi/post/dealing-with-the-meltdown-patch-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Althaf Hameez
- Published: Jan 7, 2018
- Type: Incident / postmortem

AWS infrastructure maintenance related to Meltdown patches led to severe CPU utilization spikes across Grab's ElastiCache Redis instances. Because Redis is single-threaded, spikes past 50% CPU on two-vCPU instances threatened service capacity, and initial Multi-AZ failovers only provided temporary relief until the new master nodes received rolling patches. To handle the increased overhead before their peak traffic window, the engineering team horizontally scaled both clustered and non-clustered Redis fleets. For Redis 3.2.4 clusters lacking live re-sharding support, they provisioned larger clusters, warmed caches, and redirected traffic. Non-clustered workloads were resolved by provisioning extra nodes, migrating compatible services to Redis Cluster, or updating application code to shard data across multiple instances.


### [Migrating Existing Datastores](https://yomu.fyi/post/migrating-existing-datastores.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Nishant Gupta
- Published: Aug 8, 2017
- Type: Problem & solution

Grab's Identity team faced imminent memory exhaustion on a single Redis node used to cache mobile authentication tokens under rapid user growth. Because read traffic outweighed write traffic by roughly 200 times, the team opted for an AWS ElastiCache cluster with three shards and two read replicas per shard. They executed a zero-downtime, six-phase migration plan while handling a peak load of 20,000 queries per second. The migration transitioned through initial one-time data replication, asynchronous shadow writes, synchronous dual writes, asynchronous read validation, switching primary reads, and final write cleanup. Controlled by feature flags and monitored with metrics at every stage, the migration completed without invalidating tokens or causing service disruptions.


### [How We Scaled Our Cache and Got a Good Night's Sleep](https://yomu.fyi/post/how-we-scaled-our-cache-and-got-a-good-night-s-sleep.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Gao Chao
- Published: Jun 19, 2017
- Type: Problem & solution

Growing business load on the Common Data Service (CDS) created potential bottlenecks for its single-threaded Redis cache on ElastiCache, necessitating horizontal scaling for greater capacity and throughput. After ruling out master-slave replication and intermediate Twemproxy setups due to memory constraints and proxy I/O bottlenecks, the team implemented client-side sharding. Using an internal Go package for consistent hashing, CDS instances hash cache keys locally to determine the target shard. The implementation encapsulates hashing inside a thin \`ShardedCache\` wrapper sharing the original cache interface while supporting Ketama and custom hash functions. Deploying via double-writing cron jobs during off-peak hours reduced database read pressure and improved P99 latency.


### [A Key Expired in Redis, You Won't Believe What Happened Next](https://yomu.fyi/post/a-key-expired-in-redis-you-won-t-believe-what-happened-next.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Karan Kamath
- Published: Mar 27, 2017
- Type: Incident / postmortem

Grab experienced an issue where its Unicorn API served stale data for up to 45 to 60 minutes despite expected cache invalidation times totaling around 11 minutes. The setup utilized ElastiCache Redis 2.x configured with a single master node for writes and two read-only slaves handling reads. Investigation revealed that in Redis 2.x, slave nodes do not expire keys on their own and only delete them upon receiving an explicit DEL command from the master. Because the master only actively checks and deletes 200 random keys per second, clearing expired keys across roughly 5.6 million cached items mathematically required over 110 hours, resulting in slaves serving expired data.


### [Speeding Up Sprockets for Faster Asset Pipelines](https://yomu.fyi/post/speeding-up-sprockets-for-faster-asset-pipelines.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Richard Schneeman
- Published: Feb 22, 2016
- Type: Explainer

The Rails asset pipeline can take over 20 times longer than dependency installation because Sprockets consumes memory, disk I/O, and CPU while reading, fingerprinting, compressing, and resolving dependent assets. Reusing its disk cache failed across Capistrano releases and Heroku builds because cache keys and dependency data embedded absolute paths, so identical files appeared different when compiled from changing directories. Sprockets 3.3+ addresses this with UnloadedAsset, normalized relative cache keys, and URITar, which compresses paths and URIs before caching and expands them when loading; Windows portability was fixed in 3.3.4. In an example app, the initial compile remained about 38 seconds, while the second took 3 seconds, a roughly 1,200% speed increase when compiled assets were reused.


[Newer posts](https://yomu.fyi/topic/caching/page/4.md)