Loading…
Topic
375 posts about Scalability. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
While preparing for Black Friday Cyber Monday (BFCM), Shopify’s Products team found that historical reporting records were causing MySQL to scan many InnoDB pages to return a single record. An intern built a Rails background job to delete the records safely, using Shopify’s job-iteration library to process small batches and an internal throttling enumerator to pause when database health or replication lag became a concern. Indexed fetch queries and cursor-based batching avoided repeated scans and timeouts, while the iterator deleted each batch. Run about two weeks before BFCM, the job deleted more than 5 billion records, peaked at approximately six million records per minute, and released three terabytes of storage. Afterward, the database scanned around three times fewer pages per returned record, improving resilience to flash-sale traffic and producing large cost savings.
Grab previously managed experiments using custom service-level code and a toggling library that queried a shared Redis instance, creating latency risks and a single point of failure across backend microservices. To achieve reliable, sub-microsecond feature evaluations, the team designed a Go SDK that resolves rollouts and A/B tests entirely in memory without runtime network I/O. Backend services periodically poll JSON-defined configuration schemas stored in Amazon S3 through a Universal Configuration Manager. The SDK evaluates contextual attributes called facets locally and pushes decision telemetry asynchronously to an S3 and Presto data lake. This architecture allows engineering and product teams to gate deployments and run server-side experiments safely without service disruption.
As Grab migrated from a monolith to hundreds of microservices, managing global rate limiting became essential to prevent cascading failures and resource exhaustion. To avoid putting a rate limiting service on the critical path of every API call, Grab built Quotas, an asynchronous rate limiting system. Client services use a lightweight SDK and middleware to read rate limiting decisions from local in-memory caches and stream usage metrics asynchronously via Apache Kafka. The Quotas service aggregates usage data locally, flushes stats to Redis periodically, and publishes updated rate limiting decisions back over Kafka topics. In production, Quotas successfully handles 200k peak transactions per second with decision enforcement delays capped at 200 milliseconds.
Grab Senior Data Scientist Dr. Liuqin Yang, Professor Defeng Sun, and Professor Kim-Chuan Toh received the 2018 Beale-Orchard-Hays Prize for their research paper introducing SDPNAL+. The software employs a majorised semismooth Newton-CG augmented Lagrangian method to solve large-scale semidefinite programming problems with nonnegative constraints. While traditional methods struggled beyond matrix dimensions of 2,000 and 5,000 constraints, SDPNAL+ successfully scales to matrix dimensions of 9,261 and over 12 million constraints. In benchmark testing, the software solved a problem on a desktop PC in 1.5 hours that required 122 hours on a 56-core CPU and 128-GPU cluster using a traditional solver. Grab implements these optimisation techniques to accelerate its passenger-driver allocation algorithms by hundreds of times.
Grab built its internal Experimentation Platform (ExP) to replace a manual, expensive testing process that required bespoke meetings, custom logging pipelines, and service modifications for each experiment. ExP provides a unified infrastructure featuring a centralized management UI, automated real-time data streaming to S3, and SDKs for Android, iOS, and Go. The platform leverages JSON-based experiment definitions delivered through dynamic configuration management, enabling client-side evaluation without costly network calls. It addresses marketplace network effects and inter-experiment interference through mechanisms such as geo-temporal segmentation and domain-layer models. The platform has scaled to run approximately 25 concurrent experiments while computing roughly 2,500 metrics and 50,000 experiment-metric combinations daily.
Shopify outgrew its ability to scale by buying larger database servers, so it introduced database sharding in 2015. Sharding improved performance and horizontal scalability but made actions dependent on every shard, increasing the risk of platform-wide unavailability. In 2016, Shopify reorganized its runtime around pods: each pod groups shops on isolated datastores, while shared workers, application servers, and load balancers are restricted to one pod at a time. Sorting Hat assigns each request to a pod and location, while every web request and delayed job is kept within one pod; paired data centers and Pod Mover support recovery without dropping requests or jobs. The design makes a pod the unit for scalability, resiliency, and disaster recovery, and simplifies evacuating an entire data center.
ipify began as a free API that returns a caller’s public-facing IP address, created to help discover cloud instances’ external addresses without management APIs. Its initial implementation was a sub-50-Loc Node service deployed on one Heroku dyno for about $7/month; rising usage caused slow responses and 503s. Investigation showed roughly 10 rps, while one Node process and string manipulation around X-Forwarded-For limited throughput; clustering doubled capacity, and the implementation later used Go. Heroku supplied load balancing, automatic restarts, and horizontal dyno scaling as traffic expanded through bursty demand and malicious use; the operator chose manual capacity rather than paid autoscaling. The service surpassed 30B requests per month, routinely serving 2k–20k RPS with 1–20ms average response times at $150–$200 monthly; ownership later passed to whoisxmlapi.com, whose work includes an improved UI and more data endpoint.
Grab established a dedicated Growth Hacking team within its Technology organization to pursue high-risk, niche initiatives and scale impact across 68 million regional users. Positioning the team within engineering allowed rapid A/B testing and simultaneous multi-market deployments driven by structured growth loops. The team prioritizes and evaluates all initiatives using a Growth Factor metric, calculated as the increase in rides divided by the increase in costs. To improve driver engagement, the team implemented a Spin-to-Win game based on B.F. Skinner's variable ratio reinforcement principles, delivering probabilistic monetary and merchandise rewards upon meeting daily ride thresholds. Ongoing regional experiments continue to evaluate metrics including driver acceptance, cancellation rates, and driver ratings.
Kickstarter’s front-end team encountered a Redux action-namespacing problem when multiple instances of the same functional React component responded to actions intended for only one instance. They rejected local state because the application used functional components and redux-loop, which routes state updates and side effects through reducers, and they sought programmatic namespacing compatible with combineReducers and a low-surface-area component API. The team compared nested reducers, higher-order reducers, and a closure-based module pattern; the latter met the stated constraints with namespaced actions, wrapper components, reducers, and a namespace returned by a factory that defaults to a UUID. Namespaces must be preserved when reducers trigger further actions, including redux-loop results, and complex composition can lose the required binding. The approach has worked for most reusable and compound components, while possible performance costs from many reducers remain a future concern.
Growing business load on the Common Data Service (CDS) created potential bottlenecks for its single-threaded Redis cache on ElastiCache, necessitating horizontal scaling for greater capacity and throughput. After ruling out master-slave replication and intermediate Twemproxy setups due to memory constraints and proxy I/O bottlenecks, the team implemented client-side sharding. Using an internal Go package for consistent hashing, CDS instances hash cache keys locally to determine the target shard. The implementation encapsulates hashing inside a thin `ShardedCache` wrapper sharing the original cache interface while supporting Ketama and custom hash functions. Deploying via double-writing cron jobs during off-peak hours reduced database read pressure and improved P99 latency.
Grab transitioned its transportation platform from a single Ruby on Rails monolith on Amazon EC2 and Amazon RDS MySQL to a microservices architecture hosted on Amazon Web Services. The platform processes multi-petabyte real-time data flows and hundreds of millions of GPS data points to match drivers with passengers and push proactive demand heat maps. Operational efficiency is sustained with fewer than ten full-time infrastructure engineers, supported by AWS managed services. The analytics backend also transitioned from MySQL to Amazon Redshift, eventually moving to an Amazon S3 data lake using Amazon EMR and Presto. These data-driven matching systems improved driver-passenger allocation rates by up to 30%.
Grab experienced severe performance degradation, long queue times, and connection timeouts on its Amazon Redshift analytics cluster as user concurrency and reporting workloads expanded. Although an initial Amazon S3 data lake decoupled storage from compute, business users required standard SQL interfaces rather than Spark data pipelines. The team deployed Presto clusters on AWS EMR, switching their storage format from AVRO to Parquet to support ANSI SQL querying directly against S3. Utilizing a shared Hive metastore on Amazon RDS allowed Grab to adopt a shared-data multi-cluster architecture that isolated distinct workloads across dedicated compute clusters. This setup enabled rapid cluster scaling, streamlined failover, and matched Redshift performance on partitioned time-range queries.
Grab maintains a selective engineering recruitment process in Singapore, with only three to five percent of candidates ultimately receiving an offer. The evaluation pipeline typically spans three to four weeks and includes an initial HR phone screen, an online coding round, and consecutive technical interviews. Recruiters assess candidate capability via public GitHub projects, presentations, and technical blogs rather than traditional CV qualifications alone. The company prioritizes experience in low-latency distributed systems and microservices design alongside strong cultural alignment with peer collaboration.
Building client-side load balancing for Grab's Common Data Service prompted a move from AWS Elastic Load Balancers to DNS discovery due to persistent connection issues and unpredictable scaling events. After patching an open-source library that failed to rotate IP sequences properly, the author evaluated different Go patterns for round-robin routing. A mutex-protected array counter provides the simplest model for basic retrieval, though adding mutations requires careful lock coordination. Alternatively, a dedicated balancer goroutine receiving requests over nested channels enables explicit operation timeouts and centralized event handling at the cost of higher code complexity and channel creation overhead. The author recommends the mutex approach for resource fetching and the goroutine-based design for workload balancing.
Heroku’s short request timeout and H12 errors reflect a scaling problem: standard web servers process queued requests serially, so one long operation can block others. The post separates slow server-side actions, such as video processing or PDF generation, from file uploads, and recommends background processes for the former and direct browser uploads to Amazon S3 or Google Cloud Storage for the latter. When a client must wait for completion, it describes polling or a WebSocket, noting that WebSocket connections close after 55 seconds of inactivity and ordinary pages still need to respond within 30 seconds. Avoiding lengthy web-process work improves performance and scalability; rack-timeout for Ruby or expressjs’ timeout for Node can terminate requests after a few seconds, with a recommended limit no higher than 10 seconds.
The best engineering reads, once a week.
