---
title: "Caching"
description: "95 posts about Caching, summarised, each linking to the original."
---

# Caching
> 95 posts about Caching, summarised, each linking to the original.

## Articles

### [Lessons From Building iOS Widgets](https://yomu.fyi/post/lessons-from-building-ios-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify’s widget team describes a third-generation redesign for iOS and Android after the original iOS widgets offered only two metrics and consumed substantial space. The widgets target merchants who need fresh shop analytics, using native SwiftUI on iOS because WidgetKit extensions have memory constraints, serialize SwiftUI view hierarchies to disk, and lacked official React Native support. Independent widget-side fetching uses shared session-management code and tokens; repeated update callbacks caused duplicate network requests and token-refresh races, so the team introduced a short-lived, widget-specific cache. Replacing existing widgets also exposed migration limits: changing widget kind identifiers could blank or remove installed widgets, leading Shopify to deprecate old widgets while releasing new ones and guide merchants through the transition. The team planned further customization and integration with iOS 16 features, including watch and lock-screen complications and live activities.


### [Lessons From Building Android Widgets](https://yomu.fyi/post/lessons-from-building-android-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify revisited its Android widgets alongside an iOS widget effort, aiming to give merchants faster access to frequently changing business metrics. The team chose native Android development because React Native lacked official RemoteViews support, while widget configuration data was persisted with shared preferences and used to build requests. Because Android could invoke onUpdate() multiple times per cycle, they added a one-minute, short-lived cache keyed by widgetID and mindful of configuration such as locale. This reduced redundant network calls, system load, and incorrect analytics, while the widget rollout also exposed migration limitations between old fixed-size widgets and new space-responsive ones, leading Shopify to deprecate the old widgets rather than transform them.


### [RailsConf 2022: 10 Shopify Tech Talks You Might Have Missed](https://yomu.fyi/post/railsconf-2022-10-shopify-tech-talks-you-might-have-missed.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 30, 2022
- Type: Announcement

Ruby Central has posted videos from RailsConf talks and workshops online, following the 2022 in-person conference where several Shopify Ruby on Rails team members attended and around a dozen gave talks. The collection features ten Shopify sessions on Rails’s state, Ruby’s YARV virtual machine, cache serialization, algorithmic complexity, historical Ruby gems, supply-chain attacks, team trust, GraphQL delivery, and regular-expression denial-of-service attacks. Several talks use concrete technical examples: MessagePack replaced Marshal in the core monolith’s cache, OpenTelemetry is used to diagnose performance issues, and one tutorial demonstrates GraphQL delivery through WebSockets, Webhooks, WebAssembly code, and Bulk Operations. The sessions also connect implementation details with broader engineering practices, including investment in open-source teams, playful problem solving, pair programming, Rubocop, compilers, and examining older code to inform current decisions.


### [Launch Week 5 Hackathon Winners](https://yomu.fyi/post/launch-week-5-hackathon-winners.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: tyler\_shukert
- Published: Aug 25, 2022
- Type: Announcement

Supabase announces the winners of its Launch Week 5 Hackathon, which received 46 project submissions and was described as the most popular hackathon yet. The awards recognized Supabase Cache Helpers as Best Overall for a caching utility that turns Supabase queries into cache keys and provides methods to retrieve and mutate database data, while Super UI was runner-up with a customizable React component library. Other category winners included Hop, a Tinder-like job board; Otentik Authenticator, a desktop app for managing two-step verification tokens; That Hot Dog Game, which uses facial recognition to detect eating; Quid Faciam Hodie?, a video journaling app; and RepoWatch, which shows Supabase GitHub organization updates in realtime. Winning team members receive limited-edition golden Supabase caps, runners-up receive silver caps, and all submissions are available on madewithsupabase.com.


### [Slack Consolidate: a slackbot built with Python and Supabase](https://yomu.fyi/post/slack-consolidate-a-slackbot-built-with-python-and-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Rodrigo Mansueli
- Published: Aug 9, 2022
- Type: Tutorial

Slack Consolidate is a Python Slackbot that watches configured Slack channels and reposts messages to destination channels according to routing rules stored in Supabase. The tutorial sets up Slack app permissions from a manifest, installs the Python SDKs, configures environment variables for Supabase and Slack tokens, and creates slack\_channels and slack\_watcher PostgreSQL tables. Its polling loop loads channel mappings, fetches up to 20 recent messages, converts Slack timestamps to ISO-compatible values, inserts messages into the watcher table, builds message links, and posts through a bot client; it waits 1.4 seconds between channel processing and rescans channel configuration hourly. Public-channel posts contain a link, while private-channel posts include the message text as well. The project also provides a Google Colab notebook for trying the example.


### [Five more things](https://yomu.fyi/post/five-more-things.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Ant Wilson
- Published: Dec 3, 2021
- Type: Announcement

Supabase announced a series of platform updates, technical releases, and community initiatives to conclude its Launch Week. The team open-sourced pg\_graphql, an in-progress PostgreSQL extension that keeps schema generation, query parsing, and resolvers natively on the database server. Supabase also enabled a global CDN spanning over 250 cities, delivering edge caching with a default one-hour cache time for storage assets alongside TLS 1.3, HTTP/3 over QUIC, and Brotli compression. While Supabase Functions is not yet released, two public RFCs were opened to explore architecture options, leaning heavily toward container-based execution. Finally, the update introduced a free SaaS development course with egghead.io and launched the ten-day Holiday Hackdays hackathon.


### [Building Blocks of High Performance Hydrogen-powered Storefronts](https://yomu.fyi/post/building-blocks-of-high-performance-hydrogen-powered-storefronts.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 8, 2021
- Type: Explainer

Hydrogen is a React-based framework for custom Shopify storefronts, designed to balance fast loading, personalized and dynamic commerce, and developer experience. The central trade-off is that static generation and edge delivery can speed loading but defer personalization, while conventional server-side rendering can slow time-to-first-byte as data is fetched. Hydrogen addresses this with React 18 alpha streaming SSR powered by Suspense, which streams the page shell before data resolves and progressively hydrates components without extra client round trips. React Server Components keep server-only code out of bundles, support server-side data access, and enable component-level updates that preserve client state. Built-in fetching and configurable caching, including stale-while-revalidate, work with Oxygen’s colocated runtime, while developers can mix server rendering, client fetching, and edge delivery at page or component level.


### [The Vitality of Core Web Vitals](https://yomu.fyi/post/the-vitality-of-core-web-vitals.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 4, 2021
- Type: Tutorial

Core Web Vitals were introduced by Google in 2020 as unified guidance for web UX, and their loading, interactivity, and visual stability metrics became one factor in ranking systems after rollout from mid-June through the end of August 2021. The article explains LCP, FID, and CLS, including how LCP identifies the largest visible element, FID measures delay before the browser responds to a first interaction, and CLS sums unexpected layout shifts. It gives good and poor thresholds of 2.5 seconds and 4 seconds for LCP, 100 milliseconds and 300 milliseconds for FID, and 0.1 and 0.25 for CLS, with intermediate scores needing improvement. Recommended improvements include updating dependencies and tree-shaking unused library code, preloading critical assets, improving server response time, lazy-loading below-the-fold content, compressing images, and reserving image space to prevent shifts.


### [Making Shopify’s Flagship App 20% Faster in 6 Weeks Using a Novel Caching Solution](https://yomu.fyi/post/making-shopify-s-flagship-app-20-faster-in-6-weeks-using-a-novel-cachi.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2021
- Type: Problem & solution

Shop’s home feed, the app’s most used feature, aggregated orders from millions of Shopify and non-Shopify merchants plus tracking data from dozens of carriers, accounted for 30% of database load and affected application performance. Before each database update, the custom write-through cache marks a user’s cache invalid, applies the write, then refreshes and revalidates it. To handle concurrent updates, it uses a separate pending-writes key and a short-expiry key to distinguish active writes from interrupted processes, while Active Record Concerns preserve the existing ORM interface. After a staged validation rollout, the cache reduced database load by 15%, overall app latency by about 20%, and overall GraphQL latency by 20%, with a double-digit decrease in CPU usage.


### [Last Mile Redis](https://yomu.fyi/post/last-mile-redis.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jul 16, 2021
- Type: Problem & solution

Last Mile Redis presents a globally distributed caching design aimed at keeping backend responses under the 100-millisecond threshold by placing Redis near users. It argues that regional datasets overlap less than expected, making one Redis server per region a practical alternative to centralized sharding. The design combines standalone regional caches with a Dallas primary Redis and writable replicas configured with replica-read-only no, allowing local writes while replication distributes global changes. A DEL issued at the primary can therefore purge bad cache data across regions, while timestamped keys and expiration handle ordinary freshness; the same mechanism can also broadcast content or fan out queue messages.


### [Docker without Docker](https://yomu.fyi/post/docker-without-docker.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Apr 8, 2021
- Type: Explainer

Fly.io explains how it accepts OCI container images while avoiding Docker at runtime, choosing Firecracker micro-VMs because Docker’s isolation is not strong enough for its high-density multitenant environment. It describes OCI images as ordered tarball layers, showing how registry authentication, manifests, architecture selection, blob downloads, and configuration data are enough to reconstruct a container filesystem and entrypoint. On the platform side, flyctl uses Docker’s client library to push images, while a custom Go repository server adds token-based authorization, repository rewriting, and handling for cross-repository blob mounts and registry state. Servers use containerd as a cache over an LVM2 thin pool, leases and copy-on-write snapshots to provision VM filesystems quickly, and a Rust init configures mounts, networking, DNS, SSH, and the entrypoint before Firecracker runs the workload.


### [The 5-hour CDN](https://yomu.fyi/post/the-5-hour-cdn.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Mar 16, 2021
- Type: Tutorial

The post presents a five-hour path to building a functional content delivery network as a distributed caching proxy rather than a Google-scale infrastructure project. It defines the architecture as NGINX instances deployed across cities, with DNS rules or Anycast directing users to regions and hash-based load balancing assigning URLs to cache shards. Health checks and multi-location monitoring detect failed or slow regions, while peer caches, origin-access protection, and request coalescing address redundancy, security, and cache-miss spikes. The design can improve cache ratios and reduce origin load, but cross-region proxying can add latency, and distributed invalidation remains unresolved; app-specific workloads may be better served by distributing the application itself. The conclusion is that existing NGINX features and modest scripting make a simple CDN practical, while advanced capabilities such as image optimization, WAF, rate limiting, and bot detection remain possible extensions.


### [Persistent Storage and Fast Remote Builds](https://yomu.fyi/post/persistent-storage-and-fast-remote-builds.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Feb 15, 2021
- Type: Announcement

Fly introduces persistent volumes for apps and a new remote-build path designed to make Docker image builds faster. Volumes are created through flyctl, backed by encrypted ext4 filesystems on Linux LVM2 thin pools, and attached one-to-one with app instances on compatible hosts. Volumes survive reboots and deployments and are described as performance-competitive with, usually slightly faster than, EBS, but raw-volume resilience remains the customer’s responsibility. Because volume count limits instance count and prevents an extra canary volume, volume-backed apps use rolling deployments; the post identifies replicated database clusters, caches, metrics, and restorable data as suitable cases. Remote builds now match jobs to automatically created builder apps with attached storage for cached layers and manifests; builders expose authenticated Docker servers, terminate after 10 minutes of inactivity, and are not restarted after successful exit.


### [Serving Driver-partners Data at Scale Using Mirror Cache](https://yomu.fyi/post/serving-driver-partners-data-at-scale-using-mirror-cache.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Indrajit Sarkar
- Published: Jan 26, 2021
- Type: Problem & solution

Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.


### [How we sped up transformer inference 100x for 🤗 API customers](https://yomu.fyi/post/how-we-sped-up-transformer-inference-100x-for-api-customers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nicolas Patry
- Published: Jan 18, 2021
- Type: Problem & solution

Deploying large transformer models in production poses severe latency and scaling hurdles for machine learning engineers. Hugging Face achieved a cumulative 100x inference acceleration on its Accelerated Inference API by pairing library-level algorithmic adjustments with low-level hardware compilation. The initial 10x improvement reduces forward-pass computations, such as restricting attention calculations to the latest token in GPT generation tasks, while utilizing Rust-based tokenizers and caching mechanisms. The subsequent 10x speedup requires custom static graph compilation targeted to selected CPU or GPU hardware profiles, applying unused flow removal, instruction-specific layer fusion, and precision quantization via ONNX Runtime. These coordinated techniques enable real-time prediction latency across varied model sizes and demand profiles without sacrificing task accuracy.


### [Making the Supabase Dashboard Supa-fast](https://yomu.fyi/post/making-the-supabase-dashboard-supa-fast.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Inian Parameshwaran
- Published: Dec 13, 2020
- Type: Problem & solution

Adding features to the Supabase single-page application dashboard risked performance regressions due to increasing JavaScript bundle sizes. To establish a baseline, the engineering team introduced next-bundle-analyzer and tracked Real User Monitoring alongside Core Web Vitals using Sentry. Optimization efforts focused on pruning dependencies, which included swapping Moment.js for day-js, replacing Joi with ajv, reverting crypto-js to version 3.3.0, and moving the 388 KB zxcvbn password module to a backend API. The team also implemented Next.js dynamic imports for heavy components like spreadsheet parsing, removed legacy server-side props to unlock Automatic Static Optimization, and configured long cache headers on assets. These architectural adjustments brought Core Web Vitals within recommended thresholds and lowered client-side page transition payloads below 200 KB of JavaScript.


### [Simplify, Batch, and Cache: How We Optimized Server-side Storefront Rendering](https://yomu.fyi/post/simplify-batch-and-cache-how-we-optimized-server-side-storefront-rende.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 10, 2020
- Type: Explainer

The post explains how a new Ruby-based server-side Storefront Renderer reduced the time required to serve Shopify storefront requests. It combines MySQL multi-statement queries, handcrafted SQL, a thin data-mapping layer built from plain old Ruby objects, query book-keeping with eager- and lazy-loading, multiple LRU caching layers, and techniques for reducing memory allocations. For a product page, one database round trip can load the product, variants, images, shop, theme, and related resources; later requests can replay previously observed queries early, while less frequently used data remains lazy-loaded. The resulting renderer serves 75% of requests in under ~45ms, 90% in under ~230ms, and 99% in under ~900ms, with average response time nearly five times faster than the previous implementation.


### [Building Mental Models of Ideas That Don’t Change](https://yomu.fyi/post/building-mental-models-of-ideas-that-don-t-change.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 22, 2020
- Type: Explainer

Amid a constant stream of frameworks, languages, and platforms, the post argues that learning enduring principles is more useful than trying to absorb every new tool. It presents mental models as a checklist for prioritizing concepts, recording what matters, and making more informed decisions across engineering and management. Engineering examples include alerting on silent failures, queuing background jobs for faster responses and retries, using caching and denormalization for read-heavy workloads, and sharding or deliberate design choices for write-heavy systems. The post also recommends horizontal scaling for long-term growth, then turns to practices such as aligning incentives, clarifying product purpose, building rapport before feedback, and operating a few months ahead. The stated aim is to use these models to make more informed decisions rather than chase an ever-expanding list of new technologies.


### [How Shopify Reduced Storefront Response Times with a Rewrite](https://yomu.fyi/post/how-shopify-reduced-storefront-response-times-with-a-rewrite.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 20, 2020
- Type: Problem & solution

Shopify rewrote the server-side Storefront Renderer, which loads Liquid themes and storefront data before returning HTML, because the legacy Rails-monolith implementation had developed stricter performance demands and rising time-to-first-byte as traffic grew. The new single-purpose application separates storefront traffic from checkout, admin, and API traffic, uses active-active replication with dedicated read replicas, and adds mechanisms for high-load resilience. During migration, a Ruby verifier compares status codes, headers, and bodies from both implementations, while a custom Lua module on OpenResty samples production traffic and routes requests based on verification results. The rollout had reached more than 90% feature parity, and the new implementation averaged 4x faster server response times, with ongoing work aimed at full parity and retiring the legacy system.


### [Returning 575 Terabytes of Storage Space to Our Users](https://yomu.fyi/post/returning-575-terabytes-of-storage-space-to-our-users.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Lucas Nelaupe
- Published: Feb 25, 2020
- Type: Problem & solution

Android Vitals data revealed that 15.7% of Grab users had less than 1GB of free device storage and uninstalled the app at 1.2 times the normal rate. To understand on-device storage consumption, the team instrumented session launches using the Android StorageManager API to collect binary size, cache folder size, and total footprint metrics. Analysis showed unusually large cache sizes driven by orphaned cache folders from discontinued third-party libraries, including an image library replacement of Picasso by Glide. An automated cleanup routine deployed in app updates purged legacy cache directories upon launch. This mechanism reclaimed 575 terabytes of junk data across more than 13 million devices, averaging 40MB per user.


[Newer posts](https://yomu.fyi/topic/caching/page/3.md) · [Older posts](https://yomu.fyi/topic/caching/page/5.md)