Loading…
Topic
375 posts about Scalability. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Payment processor Xendit needed a fast, cost-effective counter-fraud watchlist system to screen transactions against international sanctions lists in real time without relying on expensive providers like Worldcheck or Refinitiv. Leveraging their existing familiarity with Postgres, the team loaded parsed sanctions lists from the United Nations and the Indonesian government into Supabase. They implemented full-text search using the Postgres Trigram extension, enabling relevance scoring across queried records. A dedicated database function handles the search queries directly via Xendit's Python clients, allowing the entire solution to reach production in under one week. The deployment has operated reliably in production for nine months without issues, with potential future iterations planned around machine learning techniques.
Grab's legacy system stored booking and fare details in a single relational table, creating a bloated booking entity that tracked only the latest fare state and hindered rapid feature iteration. To resolve scalability, stability, and debugging challenges across millions of daily bookings, the team developed Fare Storage using the Event Sourcing pattern. The new architecture persists all fare modification events chronologically in DynamoDB, backed by a cache for eventually consistent reads and message streaming for downstream processing. The platform employs optimistic locking with versioning to manage concurrent updates, enforces idempotency through client-generated transaction UUIDs, and delegates metadata serialization to an SDK to prevent storage API changes.
Supabase has launched Supabase.js 1.0, updating its client library ecosystem with breaking API changes, scalability enhancements, and new documentation tooling. The release introduces native TypeScript implementations across all underlying sub-libraries, including supabase-js, postgrest-js, gotrue-js, and realtime-js, which also enables auto-generated reference docs. In response to community feedback, database and authentication operations now return structured error objects alongside data payloads instead of throwing runtime exceptions. Realtime client scalability is improved by consolidating multiple active subscriptions into a single socket connection per Supabase client. Additionally, the release updates authentication workflows via the newly created gotrue-js library to support third-party logins and OAuth providers alongside revised method names.
Amid a constant stream of frameworks, languages, and platforms, the post argues that learning enduring principles is more useful than trying to absorb every new tool. It presents mental models as a checklist for prioritizing concepts, recording what matters, and making more informed decisions across engineering and management. Engineering examples include alerting on silent failures, queuing background jobs for faster responses and retries, using caching and denormalization for read-heavy workloads, and sharding or deliberate design choices for write-heavy systems. The post also recommends horizontal scaling for long-term growth, then turns to practices such as aligning incentives, clarifying product purpose, building rapport before feedback, and operating a few months ahead. The stated aim is to use these models to make more informed decisions rather than chase an ever-expanding list of new technologies.
Zoom announced an updated V2 REST API Rate Limiter launching in March 2020 to replace its previous one-size-fits-all throttling system. Rather than applying uniform constraints across all subscription tiers, the new framework cross-references customer Account Types against API Request Groups categorized by infrastructure importance and load intensity. While most API rate limits are increasing, resource-intensive Dashboard APIs are reduced from 1 request per second to 6 or 12 requests per minute depending on account level. Throttled HTTP 429 responses will return X-RateLimit-Reset timestamps for per-minute limits and ISO8601 Retry-After timestamps for per-day limits. Additionally, the system introduces lock-key restrictions on specific user identifier path variables to restrict concurrency and enforce daily meeting creation caps.
Developers frequently use ngrok to establish fast, introspectable tunnels to localhost when building Zoom Marketplace applications instead of configuring complex reverse proxies like NGINX or Apache. However, using basic ngrok tunnels in production exposes systems to short URL expiration windows, single points of failure, and scalability bottlenecks. To maintain application availability, developers can purchase an ngrok license to secure a service-level agreement and support. Additionally, teams should reserve dedicated subdomains rather than using auto-generated endpoints, run ngrok as a monitored background service on a cloud provider or data center, and place a load balancer in front of the tunnel to handle heavy traffic.
Grab experienced severe system strain when sudden localized spikes in ride demand, triggered by events like heavy rain or concert dismissals, coincided with driver shortages. These localized bursts overloaded the platform and degraded the experience for users outside the affected areas. To mitigate this, engineers created the Spampede filter, a circuit-breaker mechanism placed at the start of the booking pipeline. The filter converts pickup locations into Geohash Integer buckets and partitions time using Unix timestamps, tracking unfulfilled requests in Redis with atomic increments and time-to-live expirations. When unallocated requests exceed configured thresholds within a specific bucket, the system immediately short-circuits new incoming bookings to protect overall platform stability.
Grab's Trust/Identity/Safety team built Griffin, an in-house anti-fraud risk rule engine designed to process billions of daily predictions across multiple business verticals. Initially, Grab managed fraud rules directly within backend service code, but escalating rule complexity, tight rule interdependencies, and translation gaps between data scientists and developers caused deployment delays and misfiring errors. To overcome the limitations and steep learning curves of third-party engines like Drools, the team separated the workflow into data orchestration and rule-based prediction. Griffin enables analysts and data scientists to author Python-based rules directly via a web portal and reload updated logic into memory without manual developer intervention. By eliminating I/O during rule evaluation and leveraging Gunicorn multi-processing, the engine handles over 100,000 queries per second at peak on six EC2 instances with single-prediction latencies under six milliseconds.
Grab launched an automated Electronic Road Pricing (ERP) fare calculation feature in Singapore to eliminate the need for driver-partners to manually track gantries and enter toll charges. Because Singapore gantries frequently adjust fares based on time and road conditions, manual entry often caused driver errors and revenue loss. Grab solved this by mapping precise geographical coordinates for every toll gate using satellite imagery and open data, matching frequent driver GPS pings against road layers and gantry locations. The engineering and operations teams also built an internal ERP Workflow tool to map ride trajectories and resolve driver dispute feedback within an average of one day. Following its rollout in Singapore, Grab began testing and planning regional expansion to Indonesia, Thailand, Malaysia, and the Philippines.
Grab needed a scalable logging platform to replace slow, fragmented systems that hindered debugging across their growing service fleet. Generating 25TB of daily logs, the team built a horizontally scalable Elasticsearch cluster configured via Ansible and monitored with Datadog. Although the initial proof of concept assigned all node roles (ingest, coordinator, master, and data) to every machine, operating at scale introduced major challenges with JVM heap exhaustion and cluster stability. The team resolved memory pressure and performance bottlenecks by tuning circuit breakers, lowering field data cache limits, adjusting shard allocations based on segment memory, and disabling translog compression during shard transfers.
As machine learning adoption expanded at Grab, individual teams created fragmented model serving solutions that duplicated engineering effort and required data scientists to handle underlying infrastructure. To resolve these inefficiencies, Grab developed Catwalk, a self-service machine learning model serving platform. The system runs TensorFlow Serving containers across a managed Kubernetes cluster integrated with Grab's observability stack. Data scientists deploy or update models simply by saving files using the tf.saved_model API to dedicated Amazon S3 buckets, while Kubernetes automates orchestration, ingress routing, and pod autoscaling. Catwalk abstracts server management away from data scientists, shortens deployment timelines, and provides high availability during model version rollouts.
Software systems require mechanisms beyond retries to maintain resilience during downstream outages and high traffic. Bulkheading isolates failures across infrastructure, processes, thread pools, and connection limits, preventing a single failing component from degrading an entire system. Load balancing distributes traffic across backend pools via proxies, client-side libraries, lookaside services, or sidecars, often pairing with health checks to eliminate single points of failure. When operations fail unrecoverably, fallback strategies like silent failures, local defaults, stale cache reads, and dedicated backup services enable graceful degradation. Organizations like Grab implement these approaches using internal client-side load balancers backed by etcd, cache fallbacks in microservice frameworks, and redundant core backup services.
Distributed systems that rely exclusively on retries and circuit breakers face severe failure risks, including retry storms and reliance on client-side configuration accuracy. Implementing server-side rate limiting serves as a critical defensive layer to safeguard services across evolving architectures. Throttling thresholds can be layered across per-client, per-endpoint, and server-wide granularities using algorithms such as leaky bucket or sliding windows. While local instance-level limits fail when downstream bottlenecks like databases saturate under horizontal scaling, global rate limiting coordinates traffic enforcement across entire service pools. Centralized rate limiters require asynchronous communication and fallback mechanisms to avoid becoming single points of failure or adding request path latency.
Transport demand spikes during regular commuting hours often outpace driver availability, resulting in passenger wait times and fare surges. To mitigate these imbalances, Grab created the Travel Trends Widget for its mobile feed to redistribute ride requests toward off-peak windows. The widget uses machine learning forecasting to present historical supply-demand patterns alongside pricing trends for the upcoming two hours. To handle anticipated high query rates across millions of database entries, engineers periodically load precomputed trend data into an in-memory data structure rather than querying the database per request. The feature rolled out to feeds in Singapore and Jakarta within four weeks of initial development.
Grab evolved its real-time data ingestion pipeline after an initial architecture built on Spark Streaming and Python encountered operational complexity, node failures, and data loss from S3 eventual consistency. Because the streaming workload primarily handled event partitioning and ORC file generation, the team consolidated these tasks directly into an existing Golang processing service. They implemented sharded concurrent maps for high-throughput partitioning and optimized heap allocations to resolve memory bottlenecks. This refactor removed intermediate Avro conversions and intermediate storage hops. The simplified Go pipeline eliminated data loss and reduced processing lag from up to 13 minutes down to approximately 1 minute.
Grab measures ride-hailing supply and demand across space and time to resolve geo-temporal allocation mismatches between moving drivers and ride-seeking passengers. The analytics pipeline defines supply as idle online drivers and demand as passengers checking fares within brief time slots, aggregating locations into geohashes. Each driver is mapped across neighbouring demand units and inversely weighted by straight-line distance, which yields the effective supply, supply-demand ratio, and supply-demand difference for each geographic polygon. Grab uses these aggregated metrics to identify marketplace imbalances, deploying driver heatmaps to shift excess supply and passenger travel trend widgets to defer time-insensitive ride requests.
Controlled online experimentation across diverse product verticals requires tracking interactions across systems to prevent local optimizations from causing global degradation. Grab built a batch data pipeline to capture, ingest, and process petabytes of event data to support its experimentation platform and analytics stakeholders. The architecture loads ingested event data from Amazon S3, transforms and sorts it, and writes partitioned output back to S3 with metadata registered in Apache Hive. Using Apache Spark on AWS Elastic MapReduce with Apache Airflow for orchestration, the system handles roughly 400,000 incoming events per second. The data is partitioned by event type and ingestion time and stored in Apache ORC format to streamline query workloads and reduce retrieval overhead.
Retries enable software systems to recover from transient upstream failures by automatically repeating unsuccessful requests. While retrying increases the chance of request completion across multi-host setups, it consumes additional CPU and time without inherently tracking host health. Applications must selectively retry errors with a likelihood of success, such as 500 and 503 status codes, while avoiding client-side failures like 400 or 401. To manage distributed systems safely, retries require idempotent operations or cryptographic nonces, along with backoff and jitter to prevent request stampedes. Tuning retry counts, timeouts, and delays is critical to cap the worst-case consumer response time.
Grab developed TalariaDB to support real-time SQL querying over high-velocity event streams while maintaining predictable sub-second latencies and low infrastructure costs. The distributed time-series store retains only the most recent hour of data and integrates directly with Presto via its PrestoThriftService interface. Internally, TalariaDB uses the Go-based Badger key-value store to maintain an in-memory key index of metric names and timestamps while mapping columnar event payloads directly to disk. Ingestion occurs by processing pre-partitioned event batches written to Amazon S3 via SQS notifications. By combining a zero-copy decoder with parallel split evaluation across gossiping cluster nodes, the architecture scales horizontally while serving millions of events per second.
Shopify’s Production Engineering team prepares its platform and 600,000+ merchants for Black Friday and Cyber Monday traffic without interrupting service. Preparation combines controlled fault injection “game days,” synthetic load tests, and staged freezes: a feature freeze begins several weeks ahead, followed by a code freeze allowing only critical fixes during the event. The tests probe dependencies such as shipping providers and payment gateways, including latency, fallback behavior, and responses to datastore failures. After the 2018 weekend, Shopify reported nearly 11,000 orders per minute, around 100,000 requests per second for extended periods, and overall metrics about 1.8 times 2017 levels. The account concludes that fault injection, load testing, and disaster role-play improved readiness, while third-party downtime remains Shopify’s responsibility to merchants and buyers.
The best engineering reads, once a week.
