Loading…
Topic
Redis
50 posts about Redis. Every summary links to the original.
- 50 posts
- 14 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Launching Redis by Upstash
Fly.io and Upstash launched a managed Redis service positioned close to Fly.io applications, with global replicas, a free tier, and private encrypted IPv6 connectivity. The integration uses Flycast, Machines API, WireGuard networking, and PROXY protocol headers: flyctl provisions a Flycast IP, routes it through global proxies to the nearest healthy Redis instance, and lets multitenant Redis identify the customer database without exposing its password. Applications can add replicas and write forwarding through flyctl while retaining a single Redis URI; writes go to the primary and reads and writes use geographically nearest replicas. The post illustrates edge HTML caching for a Rails recipe app, reporting low-millisecond global time to first byte for cached GET requests. It also describes read-your-own-write handling, including a 500ms replication wait and fallback reads from the source of truth.
Logbook: October 29 to November 6, 2022
This weekly logbook covering October 29 through November 6, 2022 reports updates across Elixir, Postgres, Laravel Livewire, Redis, and Fly Machines. Chris McCord describes how React inspired Elixir LiveView, while revamped Elixir documentation and Livewire tutorials cover client-side pagination, grouping, sorting, and browser notifications from the server. Fly’s revised Postgres documentation distinguishes the tools it provides for provisioning and managing a cluster from the responsibilities customers retain, and points readers to managed Postgres services. The Fly Redis dashboard command opens the Upstash Redis console with instance statistics and connection instructions, while a Fly Machines preview supports defining multiple processes to run inside one container; the log also notes updated documentation and a required CLI version update.
Rate limiting with Redis
Rate limiting at Ramp must handle third-party API quotas, globally paced Celery tasks, and inbound API traffic limited by authenticated application identities rather than only mutable IP addresses. Celery’s per-worker controls were insufficient for global limits, while queue prefetching could let rate-limited tasks block workers and stall queues. The team created a general framework for comparing algorithms and evaluated fixed-window, sliding-window, leaky/token-bucket, and Generic Cell Rate Algorithm (GCRA) approaches. They selected GCRA for implementation simplicity and compute and memory efficiency, storing a theoretical arrival time in Redis and using a Redis lock to prevent races. The implementation also uses floating-point timestamps, Redis server time, key expiration, unit tests, and manual testing; it has been introduced gradually across the three use cases, with monitoring, while additional algorithms remain planned.
Typeahead Search at Nextdoor
Nextdoor built a proximity-based autocomplete service to power typeahead search and mention features across its hyperlocal platform for hundreds of millions of entities, including users and businesses. The system shards geographic data using Uber's open-source H3 geohashing library and stores prefix indexes in memory using Redis sorted sets. By adopting a Command Query Responsibility Segregation architecture, ingestion writes are processed on Redis primary nodes and replicated to read-only search nodes with under 10 milliseconds of replication lag. Dedicated APIs handle indexing, typeahead lookups, and ranking before returning hydrated results. Operating since August 2021, the service processes hundreds of millions of monthly typeahead queries while maintaining a P95 search latency below 30 milliseconds.
Test Budget: Time Constrained CI Feedback
Shopify’s core monolith contains more than 170,000 tests, and variable CI feedback times create delays and context switches as the test pool grows. The Test Infrastructure team evaluated a Test Budget approach that builds prioritized orders on top of deterministic test selection, then stops execution at a predetermined time to increase the chance of finding failures early while accepting some risk. Six criteria were compared—failure_rate, avg_duration, churn, coverage, complexity, and default/random—using Time to First Failure, APFD, and Convergence Index; historical results were stored and queried with Redis Sorted Sets. The analysis found failure_rate had the best detection rate, while churn and complexity were inappropriate; running 60% of the suite detected 80% of failures in the mean case, and 70% of the test-selection suite detected 50% at the fifth percentile.
Scaling Shopify's BFCM Live Map: An Apache Flink Redesign
Shopify’s 2021 BFCM live map had to scale from more than 1.7 million merchants while adding product trends and unique shoppers, after the prior Cricket-and-Redis backend showed volume and delivery weaknesses. The redesign placed Apache Flink on the critical path to filter irrelevant checkout events, reducing Cricket’s workload to about one percent of event volume, while cross-region sharding and active-active deployment supported availability. Flink computed new metrics, including 500 product categories ranked by sales-volume change every five minutes, while Cricket retained existing metric computation and deduplication. In load tests, the pipeline handled 1 million checkout events per second, and its jobs ran at 100 percent uptime during BFCM without backpressure or manual intervention. Redis still caused high CPU and memory pressure, producing brief arc-visual errors for a small sample of users, so Shopify plans a simpler Flink-centered design.
Last Mile Redis
Last Mile Redis presents a globally distributed caching design aimed at keeping backend responses under the 100-millisecond threshold by placing Redis near users. It argues that regional datasets overlap less than expected, making one Redis server per region a practical alternative to centralized sharding. The design combines standalone regional caches with a Dallas primary Redis and writable replicas configured with replica-read-only no, allowing local writes while replication distributes global changes. A DEL issued at the primary can therefore purge bad cache data across regions, while timestamped keys and expiration handle ordinary freshness; the same mechanism can also broadcast content or fan out queue messages.
Simplify, Batch, and Cache: How We Optimized Server-side Storefront Rendering
The post explains how a new Ruby-based server-side Storefront Renderer reduced the time required to serve Shopify storefront requests. It combines MySQL multi-statement queries, handcrafted SQL, a thin data-mapping layer built from plain old Ruby objects, query book-keeping with eager- and lazy-loading, multiple LRU caching layers, and techniques for reducing memory allocations. For a product page, one database round trip can load the product, variants, images, shop, theme, and related resources; later requests can replay previously observed queries early, while less frequently used data remains lazy-loaded. The resulting renderer serves 75% of requests in under ~45ms, 90% in under ~230ms, and 99% in under ~900ms, with average response time nearly five times faster than the previous implementation.
More Appkata Examples to Try on Fly
Two new Appkata examples show how to run selected applications on Fly: Node-RED and Redis secured with TLS. Node-RED is a GUI flow-programming environment built on Node, with modules for connecting MQTT, Redis, Graphana, and other systems, plus creating web dashboards; the guide is intended to get it running quickly. The secured Redis example uses mkcert to simplify creation of TLS certificates, extending an earlier Redis example that presented Redis as a simple service. The post also points to Community.fly.io as a place for questions, hints, and tips, noting that Fly accounts can use single sign-on to make participation easier.
Appkata - Example Apps on Fly
Appkata introduces four example backend apps for Fly, intended to show how to deploy common services while learning the platform’s deployment patterns. The examples cover Redis as a customizable, volume-backed key/value database; MinIO as a single-node, S3-compatible object store exposed through Fly’s TLS edge connections; Gogs as a private Git server using SQLite3 and Fly volumes; and Mosquitto as an MQTT broker with TLS, username/password authentication, and volume-backed message persistence. The examples emphasize adapting existing Docker images, attaching persistent storage, and exposing services securely. The post presents these as an initial set that will expand over the following weeks, alongside a Fly Community site for questions and guidance.
Fly Answers Questions - Suspend, Resume, Restart and Redis
Fly’s answers clarify how to stop an application temporarily, restart instances, configure minimum capacity, use Redis, and invoke the CLI. The fly suspend command reduces an app’s instance count to zero while preserving its status and settings; fly resume first restores one instance, checks it, and then redeploys from the previous configuration. By contrast, fly restart restarts each instance in place without triggering a deployment, while applications cannot set a minimum instance count of zero except by suspending. Fly’s Redis implementation does not support Pub/Sub, scripting, or Geo commands, and flyctl now creates a symlink to the shorter fly command as migration toward the latter continues with script compatibility in mind.
Uncovering the Truth Behind Lua and Redis Data Consistency
Grab experienced replica CPU usage spikes following service deployments in their master/replica Redis cluster, which caused failovers to spike to 100% CPU. Investigation revealed that a post-deployment Lua monitor script executed separately on both nodes and relied on non-deterministic HGETALL key ordering. Redis encodes hash objects as either ziplists or hashtables, and restoring from an RDB snapshot initializes small hashes as ziplists even if the master previously converted them to hashtables. This encoding discrepancy caused key ordering to diverge, preventing secondary data from deleting correctly and bloating dataset sizes. Grab resolved the issue by sorting the outputs of HKEYS and HGETALL within the Lua script to guarantee deterministic execution across nodes.
Run Apollo Server Close to Your Users
Run Apollo Server Close to Your Users presents an edge deployment pattern for GraphQL APIs. The post argues that, much like serving JavaScript and markup from a CDN, placing API servers and an application cache near users can improve application speed. Its approach combines Fly.io’s platform for running API servers close to users with a global Redis cache service, then uses Apollo Server’s cache capabilities and first-class Redis support. A demo GraphQL API built from the Open Library REST API accompanies the guide, with both a guide and source code available. The author frames the design through a newly adopted GraphQL perspective, emphasizing static typing and describing the resulting setup as CDN-like infrastructure for an API.
Your Circuit Breaker is Misconfigured
Misconfigured circuit breakers can leave an application effectively unavailable during a dependency outage by allowing timed-out requests to consume worker capacity. Using Shopify’s Semian implementation, the post explains how name, error_threshold, error_timeout, half_open_resource_timeout, and success_threshold govern circuit opening, recovery, and wasted utilization across failing service instances. It models timeout spikes for a single worker, notes that separate names isolate services and instances, and presents an equation for steady-state additional utilization, while noting that context-switch costs are not included. A live test closely matched the equation’s prediction. In a Rails worker with two threads and 42 Redis instances, changing half_open_resource_timeout from 0.25 seconds to 50ms and error_timeout from 2 to 30 seconds reduced modeled extra utilization from 263% to 4%, with slower recovery; an author’s edit says success_threshold does not affect steady-state utilization.
How We Prevented App Performance Degradation from Sudden Ride Demand Spikes
Grab experienced severe system strain when sudden localized spikes in ride demand, triggered by events like heavy rain or concert dismissals, coincided with driver shortages. These localized bursts overloaded the platform and degraded the experience for users outside the affected areas. To mitigate this, engineers created the Spampede filter, a circuit-breaker mechanism placed at the start of the booking pipeline. The filter converts pickup locations into Geohash Integer buckets and partitions time using Unix timestamps, tracking unfulfilled requests in Redis with atomic increments and time-to-live expirations. When unallocated requests exceed configured thresholds within a specific bucket, the system immediately short-circuits new incoming bookings to protect overall platform stability.
Five Common Data Stores and When to Use Them
The article compares five common data stores—relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues—and connects each to appropriate design needs. Relational databases use schema-bound tables, primary and foreign keys, SQL, and ACID transactions for durable business-critical data, while NoSQL databases trade strict schemas for flexibility with document and wide-column models. In-memory stores such as Redis and Memcached provide fast access and configurable eviction for caching; Redis additionally supports richer data structures and disk persistence. Full-text search engines target word and substring searches, whereas Kafka uses a distributed append-only log and producer-consumer model to move data reliably between services. The article concludes that the right choice depends on whether data must be permanent, flexible, quickly accessed, searched, or transferred.
Up to 75% Faster Maintenances with Heroku Postgres and Redis Premium Plans
Heroku describes how it minimizes downtime during routine security and patching maintenances for high-availability Heroku Postgres and Heroku Redis plans. These plans run a writable primary alongside a hidden read-only standby, and maintenance requires moving application writes safely to the standby before promoting it. Previously, Heroku changed DATABASE_URL and restarted app dynos, then waited for the old primary to shut down and replicate remaining writes, which could extend downtime during long-running transactions. The new method redirects new connections to the new primary, kills existing connections to the old primary, and relies on reconnections plus a rolling restart to prevent further writes there. Heroku reports HA maintenances now complete in 15–40 seconds instead of 60 seconds or more, and says the behavior is default for Premium, Private, and Shield plans.
Data First, SLA Always
Grab's Data Engineering team transitioned from periodic batch ETL ingestion to a real-time change data capture architecture called Trailblazer after dataset sizes exceeded the petabyte mark. The previous system caused severe JDBC timeouts and heavy CPU loads when executing chunked or full-scan queries on unindexed upstream MySQL tables. To solve this, MySQL binary logs are captured via Debezium on Kafka Connect, buffered in Kafka, and ingested into a data lake using Spark Structured Streaming. Checkpoints are decoupled from local storage and persisted in a Redis cluster to simplify ingestion offset overrides and handle ephemeral compute clusters. The system incorporates extensive health monitoring across Airflow, Datadog, and custom services to maintain stream liveliness and avoid Kafka retention breaches.
Preventing Pipeline Calls from Crashing Redis Clusters
A single Redis slave node failure caused Grab's Apollo booking service to suffer an over 95 percent call failure rate for one minute despite running a three-shard cluster with two replicas per partition. Investigation revealed that the service configured Go-Redis to route all read queries exclusively to slave nodes to offload master CPU usage. When the slave node dropped offline, batched HMGET pipeline calls failed completely because the client wrapper treated a single command failure as a failure of the entire pipeline. Furthermore, the Go-Redis client cached cluster topology and only lazily refreshed state every sixty seconds, continuing to direct traffic to the dead replica until the timer expired. Grab addressed this risk by recommending dedicated pipeline clients configured with latency-based routing to allow reads to fall back to responsive master nodes.
Reactive Programming with Salesforce Data
Salesforce Platform Events and Change Data Capture enable custom applications to react to activity in Salesforce, motivating an event-sourced design for responsive, durable integrations. The article demonstrates a Node.js app on Heroku that consumes the Salesforce Streaming API through jsforce 1.9.0 or later, whose Faye-based streaming client handles authentication, network connectivity, and message callbacks. It notes that a basic subscription loses its position on restart and cannot coordinate parallel processing, then uses Redis to persist each topic’s Replay ID and distribute messages through pub/sub or reliable queues. Because the Salesforce Streaming API lacks consumer-group delivery, a singleton stream_consumer feeds Redis while scalable web or worker processes handle work. The design also recommends a separate non-admin integration user and monitoring connection status, active clients, last message time, and processing duration.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.