---
title: "Caching"
description: "95 posts about Caching, summarised, each linking to the original."
---

# Caching
> 95 posts about Caching, summarised, each linking to the original.

## Articles

### [Scaling Nextdoor’s Datastores: Part 5](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-5.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Slava Markeyev
- Published: Mar 19, 2025
- Type: Problem & solution

Nextdoor addressed database and cache consistency issues caused by missed cache writes and concurrent read-fill operations in their look-aside architecture. While forward row versioning prevents out-of-order write inconsistencies, writer failures and race conditions during cache misses can leave stale data persisted in Redis. To resolve this, Nextdoor built a reconciliation pipeline that consumes PostgreSQL WAL replication logs with pg-bifrost, streams changes through Apache Kafka, and executes conditional deletions in Redis. The Go-based reconciler operates in two passes using a time wheel, running one pass in near real time and a second pass after a delay exceeding web request timeouts. Because conditional deletion evaluates row versions directly in the cache, the system processes change streams out of order and scales horizontally.


### [Scaling Nextdoor’s Datastores: Part 4](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-4.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Ronak Shah
- Published: Mar 19, 2025
- Type: Problem & solution

Look-aside caching systems can become inconsistent when concurrent database updates execute cache writes out of order, allowing stale data to overwrite newer modifications. To prevent these racing writes, Nextdoor introduced a unique, monotonic db\_version column to Postgres tables using database triggers that initialize version numbers on insert and increment them on update. Application updates retrieve this new version inside a transaction block and attach it as a metadata header to serialized cache values. Redis then executes custom Lua scripts, specifically set\_if\_version and del\_if\_version, to perform atomic conditional updates that reject any incoming payload with a version lower than or equal to the stored version. This serializable check ensures that out-of-order writes are dropped and the cache remains strictly aligned with the latest database state.


### [Scaling Nextdoor’s Datastores: Part 3](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-3.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Ronak Shah
- Published: Mar 19, 2025
- Type: Problem & solution

Look-aside caching with object byte serialization can cause critical compatibility failures when application versions, dependencies, or database schemas change. Serialized cache entries bound to specific runtimes risk deserialization errors during deployments, triggering thundering herd queries against the underlying datastore. To resolve this, Nextdoor replaced runtime-bound serialization like Python pickle with MessagePack to serialize Django model objects. The team achieved forward compatibility by letting MessagePack ignore unrecognized new fields in older application code, while backward compatibility relies on mandatory default values for newly added model attributes. Nextdoor prepends a ten-byte header containing format metadata and version information before writing the serialized payload to cache stores.


### [Scaling Nextdoor’s Datastores: Part 1](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-1.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Slava Markeyev
- Published: Mar 19, 2025
- Type: Explainer

Nextdoor relies on a Django backend connected to monolithic PostgreSQL databases and Redis look-aside caches. Migrating to distributed SQL datastores proved impractical because legacy business logic depends heavily on multi-table joins that could not be rewritten. Intermediate mitigations, including read replicas and data partitioning by severing foreign keys, extended infrastructure runways but left primary databases vulnerable to load bottlenecks. Furthermore, look-aside caching and replica usage introduced data staleness risks, atomicity losses across partitioned databases, and inconsistent cache-population behaviors. To address these limitations, Nextdoor initiated an architecture redesign focused on dynamic query routing to replicas, replica-driven cache hydration, time-bounded eventual consistency, and schema-resilient cache serialization.


### [Canva incident report: API Gateway outage](https://yomu.fyi/post/canva-incident-report-api-gateway-outage.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Brendan Humphreys
- Published: Dec 20, 2024
- Type: Incident / postmortem

On November 12, 2024, canva.com suffered an outage lasting approximately one hour due to cascading API Gateway failures. The disruption began when a stale Cloudflare routing rule induced severe latency and packet loss between Ashburn and Singapore during an editor asset deployment. Cloudflare's concurrent streaming mechanism consolidated over 270,000 user requests for a single delayed JavaScript chunk; when the asset finally loaded, clients simultaneously generated a 1.5 million requests-per-second thundering herd to Canva's API Gateway. This surge collided with a telemetry performance regression involving lock contention on Netty event loop threads, exhausting off-heap memory and causing the Linux Out Of Memory Killer to terminate API Gateway containers. Canva mitigated the failure by temporarily blocking all traffic at the CDN level, stabilizing replacement Amazon ECS tasks, and gradually restoring traffic under strict rate limits.


### [The science of routing print orders](https://yomu.fyi/post/the-science-of-routing-print-orders.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Constantinos Kavadias
- Published: Dec 10, 2024
- Type: Problem & solution

Canva's global print network requires selecting optimal suppliers to balance delivery times, packaging counts, and environmental emissions. To resolve these challenges before user checkout, the engineering team designed a modular routing architecture that decouples graph construction, decision logic, and path traversal. During graph traversal, the system generates action objects capturing forward paths and decision query results, which are compiled into timestamped routing logs in blob storage for asynchronous auditing. Utilizing preprocessed graph queries alongside ElastiCache, Redis, and database read replicas, the infrastructure sustains high-throughput evaluation without coupling cost logic to traversal code. As a result, print routing completes within an average of 50 milliseconds at the 99th percentile during peak usage while maintaining 99.999% data availability.


### [Data access patterns for simple Stripe integrations](https://yomu.fyi/post/data-access-patterns-for-simple-stripe-integrations.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Ben Smith
- Published: Nov 27, 2024
- Type: Explainer

The article examines how applications should store and access product data in simple Stripe integrations as requirements for security, performance, and scalability evolve. It compares using Stripe’s built-in product fields with a separate database, while explaining that publishable API keys cannot retrieve product details and that secret or restricted keys must remain server-side. A web backend or serverless function can proxy requests securely, while CloudFront caching reduces repeated calls and latency but requires an appropriate refresh cadence. For richer metadata, inventory, variants, or custom attributes, the article describes combining DynamoDB with Lambda and Stripe, using Event Destinations or webhooks to synchronize changes and weighing that flexibility against database overhead.


### [How we improved translation experience with cost efficiency](https://yomu.fyi/post/how-we-improved-translation-experience-with-cost-efficiency.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jie Zhang
- Published: Aug 5, 2024
- Type: Problem & solution

Grab faced translation quality and cost challenges in its booking chat system when tourism resumed after COVID restrictions eased. Device language settings proved unreliable for determining message language, prompting the team to use Lingua alongside booking context heuristics and third-party fallbacks for real-time detection. To replace expensive and inaccurate general translation APIs, Grab distilled a large open-source Hugging Face model into lightweight, language-specific models trained on ten million synthetic chat examples. A post-translation validation step verifies non-translatable entities such as numbers and emojis before falling back to external services if mutations occur. Caching layers were also introduced across translation paths to curb redundant on-the-fly execution.


### [Improving Shopify App’s Performance](https://yomu.fyi/post/improving-shopify-app-s-performance.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Mar 5, 2024
- Type: Explainer

Shopify’s mobile app saw performance decline during its React Native migration, prompting a year-long effort to make critical screens load under 500ms at P75 and launch within 2s at P75. The team built real-time dashboards to filter results by device and operating system, then targeted excessive initial rendering, unnecessary background updates, launch overhead, and weak cache utilization. They introduced LazyScrollView and ListSource on FlashList, deferred work outside the viewport, enabled inlineRequires, optimized a native module, batched state updates, froze background components, and improved Restyle. They also fixed a graph QL cache issue and pre-warmed critical screen data, increasing cache-first delivery to as much as 90% for key screens. By year-end, app launch P75 was 44% faster and screen-load P75 times were reduced by 59%, with performance treated as an ongoing process requiring monitoring and repeated optimization.


### [Sliding window rate limits in distributed systems](https://yomu.fyi/post/sliding-window-rate-limits-in-distributed-systems.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Naveen Kumar Jakuva Premkumar
- Published: Dec 14, 2023
- Type: Problem & solution

Marketing communications across Grab's user base risked causing notification overload and consent revocations. To enforce personalized daily and weekly frequency caps across more than 270 million users, the team addressed segment membership storage and communication rate limiting. They adopted roaring bitmaps instead of Bloom filters to compactly store user segment data without hash collisions or costly rebuilds upon deletion. For frequency capping, they chose Amazon ElastiCache for Redis over DynamoDB, executing a sliding log rate limiting algorithm directly on the cluster using Lua scripts and sorted sets. Timestamps are stored as sorted set scores, and historical data is cleaned up via eviction ranges to prevent unbounded memory growth.


### [Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Sampling](https://yomu.fyi/post/let-ai-entertain-you-increasing-user-engagement-with-generative-ai-and.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Jaewon Yang
- Published: Oct 16, 2023
- Type: Problem & solution

Nextdoor implemented a generative AI framework to improve email notification subject lines for its New and Trending emails. Using off-the-shelf ChatGPT API outputs directly resulted in marketing-like phrases, hallucinations, and lower click-through rates compared to original user-generated post snippets. To resolve this, the team prompted the language model strictly to extract authentic post phrases without rewriting, eliminating hallucinations while preserving the original tone. Nextdoor then introduced a rejection sampling mechanism featuring a dedicated reward model that predicts user engagement and accepts generated subjects only when their score exceeds the user-written baseline. This combined system lifted sessions by 1%, boosted Weekly Active Users by 0.4%, and increased advertising revenue by 1%.


### [Streamlining Grab's Segmentation Platform with faster creation and lower latency](https://yomu.fyi/post/streamlining-grab-s-segmentation-platform-with-faster-creation-and-low.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jake Ng
- Published: Aug 15, 2023
- Type: Problem & solution

Grab's Segmentation Platform previously stored user-to-segment mappings across individual rows in ScyllaDB, causing write bottlenecks during segment creation and read latencies too high for downstream consumers. To resolve these performance limitations, the team transitioned to storing segments as Roaring Bitmaps saved as single blobs in object storage. This compression strategy splits 32-bit integer user IDs into chunks across array, bitmap, and run containers based on data density, reducing a one-million-member segment to under one megabyte. An accompanying client SDK manages segment retrieval, decoding, update notifications, and least-recently-used in-memory caching. Consequently, consumers such as Grab's communications platform achieved peak throughput of 15,000 queries per second with sub-millisecond p99 read latencies.


### [Go module proxy at Grab](https://yomu.fyi/post/go-module-proxy-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jerry Ng
- Published: Jun 30, 2023
- Type: Problem & solution

Grab's 69.3 GiB multi-module Go monorepo caused commands like go get to take over 18 minutes as Git repeatedly traversed commit history, downloaded large worktrees, and overloaded their GitLab VCS infrastructure. To bypass direct VCS queries without losing automatic updates for external repositories, the team deployed the Athens Go module proxy configured in fallback network mode. They used the GOVCS environment variable to disable Git access specifically for the monorepo path, forcing Athens to fall back to its internal object storage when resolving monorepo modules. A dedicated CI pipeline pre-populates and refreshes the Athens cache whenever new monorepo modules are released. This setup reduced monorepo go get execution times to approximately 12 seconds and allowed a 70% scale-down of the Athens proxy cluster.


### [Designing with AI: Generating unique artwork for every user](https://yomu.fyi/post/designing-with-ai-generating-unique-artwork-for-every-user.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: marijana,thor\_schaeff
- Published: Apr 7, 2023
- Type: Explainer

Supabase Launch Week 7 needed distinctive, shareable tickets that could improve on earlier Launch Week designs while keeping overlaid text readable. The team experimented with Midjourney prompts and blends, then chose minimalist purple swirls and a gold overlay approach after finding generated visuals too chaotic and direct gold prompting unreliable. For each ticket, a Supabase Edge Function retrieves a cached PNG from Supabase Storage or renders HTML and CSS into an SVG with Vercel’s Satori, layering backgrounds, a GitHub profile picture, and text before uploading the result with long cache control. A Postgres view selectively exposes ticket fields, calculates referral counts and marks a ticket golden when both Twitter and LinkedIn sharing have been detected, while row-level security protects the underlying email-containing table. Supabase Realtime then sends client-side updates so users can see progress and stop listening once golden.


### [Launching Redis by Upstash](https://yomu.fyi/post/launching-redis-by-upstash.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 15, 2022
- Type: Announcement

Fly.io and Upstash launched a managed Redis service positioned close to Fly.io applications, with global replicas, a free tier, and private encrypted IPv6 connectivity. The integration uses Flycast, Machines API, WireGuard networking, and PROXY protocol headers: flyctl provisions a Flycast IP, routes it through global proxies to the nearest healthy Redis instance, and lets multitenant Redis identify the customer database without exposing its password. Applications can add replicas and write forwarding through flyctl while retaining a single Redis URI; writes go to the primary and reads and writes use geographically nearest replicas. The post illustrates edge HTML caching for a Rails recipe app, reporting low-millisecond global time to first byte for cached GET requests. It also describes read-your-own-write handling, including a 500ms replication wait and fallback reads from the source of truth.


### [Supabase Storage v2: Image resizing and Smart CDN](https://yomu.fyi/post/supabase-storage-v2-image-resizing-and-smart-cdn.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: fabrizio,inian
- Published: Dec 13, 2022
- Type: Announcement

Supabase Storage introduces image resizing, webhooks, and a Smart CDN, positioning them as components of an on-demand image delivery system. Resizing uses imgproxy and supports width, height, and contain, cover, or fill modes through JavaScript clients or direct Storage API requests; images are not enlarged beyond their original dimensions, and aspect ratio is preserved by default. The Smart CDN stores resized results using cache keys based on an Etag, tenant ID, bucket, and image name, while webhook-driven metadata updates propagate invalidations globally within about 60 seconds. Image Resizing is beta for Pro and Enterprise customers, with 100 transformed images per project at no additional cost and $5 per additional 1,000; self-hosting is free.


### [Fetching and caching Supabase data in Next.js 13 Server Components](https://yomu.fyi/post/fetching-and-caching-supabase-data-in-next-js-13-server-components.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: jonmeyers\_io
- Published: Nov 17, 2022
- Type: Tutorial

The article demonstrates data fetching and caching with Supabase in Next.js 13’s experimental app directory, focusing on Server Components and Suspense. It creates a posts table, configures a Supabase client, and uses async components to fetch data without manually managing loading states. The examples cover static generation, dynamic routes with generateStaticParams, time-based revalidation, client-side fetching, and a realtime pattern that combines server-rendered initial data with a client subscription to Postgres changes. The article notes that Next.js 13 is stable while the app directory remains in beta, and concludes that separating caching from fetching lets applications change freshness requirements without replacing their data-fetching logic.


### [Caching Without Marshal Part 2: The Path to MessagePack](https://yomu.fyi/post/caching-without-marshal-part-2-the-path-to-messagepack.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 15, 2022
- Type: Problem & solution

Rails caching relied heavily on Marshal, but cached Ruby-specific objects could break after code changes when old payloads were loaded by code without the former classes. MessagePack provided a generic binary format whose core types exclude Ruby-specific Object and instance-variable encodings, while extension types allowed production-specific serializers such as the compact Date representation. During a roughly six-month migration, the team ran both formats, prefixed MessagePack payloads with a version byte, and used encoding failures to identify unsupported cache values. To scale beyond 128 extension types, they added an Object catchall based on as\_pack and from\_pack, plus Struct and T::Struct modules whose attribute digests detect refactors and turn stale data into cache misses. After logs showed that Marshal was no longer used, the core monolith moved exclusively to MessagePack, with the Paquito gem extracting much of the migration work.


### [Caching Without Marshal Part 1: Marshal from the Inside Out](https://yomu.fyi/post/caching-without-marshal-part-1-marshal-from-the-inside-out.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 11, 2022
- Type: Explainer

Rails caching commonly relies on Ruby’s Marshal serialization, which can encode complex objects but also embeds their class names in cache entries. Shopify describes an incident in which a beta-flag refactor changed classes while old and new code overlapped during deployment; old code then failed on cached instances containing unfamiliar class names and methods, despite passing CI. The article examines Marshal through Ruby’s marshal.c implementation, covering atomic and composite types, instance variables, object references, circularity via TYPE\_LINK, and core-type subclasses via TYPE\_UCLASS. It presents MessagePack as a more compact, stricter, and controllable alternative intended to make caching safer by default, while deferring the cache migration and implementation details to the series’ next part.


### [Apollo Cache is Your Friend, If You Get To Know It](https://yomu.fyi/post/apollo-cache-is-your-friend-if-you-get-to-know-it.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 9, 2022
- Type: Explainer

Shopify’s migration from Apollo GraphQL client 2 to client 3 prompted a closer examination of the Apollo InMemoryCache after past bugs were linked to misunderstanding or misuse. The cache stores an in-memory representation of queried data for the current browser session, and its lifecycle covers fetching, normalization, updating and merging, then garbage collection and eviction. Fetch policies determine whether data comes from the cache, the network, or both; under the default cache-first policy, incomplete data triggers a network request. Normalization breaks responses into objects, assigns cache identifiers usually from \_\_typename and id or configured key fields, and stores them in a flattened structure. Automatic UI updates depend on matching identifiers and suitable query or mutation responses, while objects made unreachable by changed identifiers can remain until garbage collection removes them.


[Newer posts](https://yomu.fyi/topic/caching/page/2.md) · [Older posts](https://yomu.fyi/topic/caching/page/4.md)