Loading…
Topic
372 posts about Scalability. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
amazon.science
Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.
Inside D. E. Shaw group, a global investment and technology development firm, millions of compute workloads run monthly across its high-performance computing infrastructure. To support capacity planning and resource allocation, it collects task-level metrics for requested and utilized compute, producing millions of distinct time series that its previous observability platform could not accommodate. Engineers benchmarked databases on identical hardware and datasets; ClickHouse ingested approximately 3.5 million samples per second on one node versus roughly 480,000 for a competing solution, while similar queries completed in seconds instead of timing out. They retained InfluxDB line protocol, batched messages into ClickHouse, and built disk-based tooling to transform and stream historical data; the resulting four-server NVMe cluster stores about 68 TB compressed and sustains 530k records per second on average. Materialized views support capacity-planning dashboards, while OpenTelemetry tracing extends the platform to other observability workloads, with SQL migration and data modeling requiring engineering effort.
Traditional time-series databases like Prometheus experience write amplification, memory overhead, and operational challenges when high-cardinality label combinations churn rapidly. ClickHouse alters these economics by abandoning dedicated series objects in favor of a wide events model, storing telemetry as rows in columnar tables with dynamic Map attributes and metric measurements. Dynamic attributes can be handled using sharded Map types alongside materialized columns and text indexes for frequently queried labels. This design trades ingestion overhead for query-time processing, relying on columnar compression, partition pruning, and parallel scans to compute aggregations on demand. ClickHouse avoids the ingest-side bottlenecks of high cardinality, though Prometheus remains preferable for moderate cardinality, native PromQL semantics, and single-series lookups.
Observability systems frequently encounter high cardinality when metrics contain numerous unique label combinations across multidimensional cloud infrastructure. In Prometheus, every unique label combination instantiates an independent time series that incurs metadata overhead, requires in-memory memSeries registration, and continually expands inverted index posting lists. The ingestion engine stores samples in two-hour Head blocks using compressed 120-sample chunks with XOR-based encodings, but rapid series creation and elevated scrape frequencies multiply chunk allocations and memory consumption. When executing read operations, broad aggregations and regular expressions require intersecting expansive posting lists and decoding entire chunks sequentially because the engine cannot push value predicates directly into compressed storage blocks. Consequently, managing high-cardinality workloads in series-oriented architectures forces operational trade-offs between system stability and granular visibility into ephemeral components.
ClickHouse Cloud provides an in-memory Join table engine for precomputed hash-map lookups and joins, aimed at enriching append-only fact data with dimensions that change occasionally. With Join(ANY, LEFT, key), inserting another row for a key acts as an upsert, while joinGet and matching JOIN clauses use the in-memory structure. Open-source Join tables are not distributed and persist each INSERT batch as separate .bin files without automatic compaction, but ClickHouse Cloud transparently uses SharedJoin with MergeTree for ALL or ReplacingMergeTree for ANY. In the demonstrated enrichment query, duplicate inserts for id 1 leave value 42 visible, while the underlying table retains duplicates until merging; the in-memory load applies FINAL and always selects the latest entry, making ANY LEFT suitable for frequently updated dimensions.
Shopify replaced its Redis-based inventory reservation system with MySQL to unify reservations with the inventory ledger and provide ACID coordination between payment and inventory updates. The new design stores one row per sellable unit in a bounded pool of up to 1,000 rows per item/location combination, uses MySQL 8's SKIP LOCKED to avoid waiting on locked units, and replenishes depleted pools. Composite primary keys reduced row locks, READ COMMITTED avoided gap locks that interfered with replenishment, and consistent lock ordering prevented deadlocks. The investigation found connection-pool usage elsewhere in checkout, rather than reservation query speed alone, was the immediate ceiling; cleanup removed 50% of primary-database reads and 33% of transactions, while configuration changes added headroom. Shopify validated the design in shadow mode, then rolled it out gradually with a Redis rollback path, meeting high-throughput targets during peak 2025 traffic.
Avride’s expanding autonomous passenger vehicle and delivery robot fleet generated thousands of data points per minute, overwhelming its legacy Apache Iceberg and Parquet metadata indexing architecture. Iceberg’s optimistic concurrency model struggled with parallel writes, pushing ingestion latencies to hours or days, inflating query times to 10–20 seconds, and requiring raw data duplication across storage locations. To resolve these bottlenecks, Avride built a custom indexing and metrics warehouse powered by ClickHouse Cloud on AWS backed by Amazon S3 object storage. The new architecture dropped ingestion delays to seconds, reduced worst-case lookup latencies to under one second, eliminated the need for bespoke caching layers, and removed payload duplication. Beyond operational analytics, the unified ClickHouse platform now supports internal ride-replay debugging tools, simulation workflows, and high-cardinality vehicle C++ performance trace profiling.
MongoDB announced the general availability of MongoDB 8.3, designed to support high-throughput AI workloads without requiring application code changes. Compared to version 8.0, the release provides a 35 percent boost in write throughput, a 45 percent increase in read speeds, and 15 percent faster ACID transactions. Additionally, cross-region connectivity for AWS PrivateLink is now generally available, enabling private traffic routing between Atlas clusters across distinct AWS regions without public internet exposure. MongoDB Atlas supports deployments across 130 regions spanning AWS, Google Cloud, and Microsoft Azure, including multi-cloud cluster topologies. These infrastructure updates target enterprise demands for sub-100ms retrieval, sub-second context updates, zero downtime, and strict data residency compliance.
amazon.science
Amazon's middle-mile logistics network coordinates tens of millions of packages across hundreds of facilities while managing subtle demand variations and major operational disruptions. To solve this mixed-integer optimization challenge without combinatorial explosion, Amazon models routes at coarse time resolutions augmented by precomputed 15-minute timing bounds. The system uses Monte Carlo methods alongside a graph attention network featuring interconnected site graphs and origin-destination graphs to capture complex spatial correlations and flow interdependencies. By generating synthetic flow permutations, planners stress-test candidate network designs against hundreds of plausible scenarios to prioritize routing optionality. This risk-aware modeling approach preserves network flexibility, absorbs severe shocks, and yields potential cost savings while safeguarding delivery promises.
Agentic analytics workflows introduce bursts of concurrent, low-latency analytical queries over continuously ingested data. Analytical query efficiency relies on columnar storage and chunk-level pruning, which require contiguous, sorted data to avoid full-table scans. Rather than evaluating warehouses solely at query runtime, this benchmark compares the write-side cost of maintaining query-ready data in ClickHouse and Snowflake during continuous ingestion at approximately one million rows per second. Snowflake relies on post-write clustering to organize data, whereas ClickHouse constructs sorted layouts on the write path and refines them via background merges. Across 100B, 200B, and 300B row checkpoints on the ClickBench dataset, ClickHouse achieved a query-ready layout at 22 times lower cost and delivered 28 times better write-side cost-performance.
amazon.science
Amazon and its vendors face an asymmetric information problem when coordinating purchase orders across fulfillment centers because neither party shares proprietary cost, capacity, or demand data. To solve this coordination gap without exposing sensitive information, Amazon's Supply Chain Optimization Technologies organization designed Flo Pro by integrating the Vickrey-Clarke-Groves mechanism with the Consensus Planning Protocol. The distributed optimization framework uses the alternating-direction method of multipliers to coordinate iterative agent responses, calculating cost-benefit transfers across rolling six-week planning horizons. In lower-dimensional settings, Amazon also evaluated a transparent menu-of-contracts approach where vendors select optimal plans priced at Amazon's incurred costs. A nine-week pilot with a consumer-product manufacturer confirmed that Flo Pro successfully aligns incentives and achieves measurable cost savings.
Blockchain gaming platform Gala struggled to ingest and query expanding volumes of telemetry and blockchain data on its Databricks-based data infrastructure, resulting in multi-minute query times and heavy engineering maintenance overhead. To solve these bottlenecks, the company migrated its data platform to ClickHouse on AWS, ingesting data through Kafka, Airbyte, Amazon S3, and Fivetran. Gala leveraged ClickHouse's indexing capabilities alongside saved queries and API endpoints to power company-wide Metabase business intelligence dashboards for non-technical teams. Following the migration completed in December 2024, data capacity expanded from 3 TB to 9 TB while query times dropped from minutes to sub-second on previously unoptimized tables. The switch also lowered initial infrastructure costs by 30 percent, with plans underway to implement ClickHouse ClickPipes for improved data processing pipelines.
Pinterest deployed a root-leaf architecture for online machine learning serving to decouple CPU-based feature preprocessing from GPU-based inference. Because the root cluster retrieved a union of features across models and fanned out the full set to every leaf, network bandwidth became a severe bottleneck. After initial tests with fbthrift LZ4 compression yielded a 20% bandwidth reduction at the cost of CPU and latency overhead, the team implemented Feature Trimmer. This mechanism uses exported model signatures as the source of truth to prune unused features on the root before transmission. By dispatching only required features per model, the system reduced network bandwidth, decreased latency, and enabled cluster rightsizing that saved over $4M annually.
Qonto struggled with slow queries, system instability, and aggressive data sampling when handling distributed traces in Grafana Tempo. To enable wide-event querying without strict cardinality limits, the team migrated their telemetry storage to ClickHouse Cloud. Telemetry from applications, Kubernetes, and GitHub routes through OpenTelemetry collectors over AWS PrivateLink into ClickHouse, while Grafana remains the primary dashboard interface. By compressing 231 TB of uncompressed trace attributes down to 376 GB, Qonto achieved substantial storage savings and integrated an AI incident companion using the ClickHouse Model Context Protocol. The new architecture eliminated cardinality policing, allowed product and support teams to self-serve incident investigations, and enabled real-time streaming pre-aggregations using Apache Flink.
ClickHouse announced several product updates and integrations designed to deepen native compatibility with Google Cloud. Migration of ClickHouse Cloud to Google Axion ARM-based processors demonstrated a 30 to 55 percent query speed increase and reduced data load times from 64 to 26 seconds on ClickBench benchmarks. An integration with Google Lakehouse Runtime Catalog enables querying Iceberg tables on Google Cloud Storage via the DataLakeCatalog engine without data movement. Furthermore, Bring Your Own Cloud reached general availability on Google Cloud Marketplace, and ClickPipes added an unordered mode using Pub/Sub notifications for GCS ingestion. Google Antigravity integrated ClickHouse through an MCP server to translate natural language into SQL and generate interactive React charts.
ClickHouse Cloud transitioned its Google Cloud infrastructure from x86 N2D instances to Arm-based Google Axion C4A instances to resolve CPU throttling and performance ceilings under peak query loads. The migration leverages Axion's dedicated physical cores per vCPU, higher memory bandwidth, local SSDs, and improved thermal and clock frequency stability. In automated ClickBench testing across 100 million rows, every query in the 43-query benchmark improved by 30% to 55%, while data load times decreased by approximately half. A production analysis of 38 migrated customer services demonstrated an average 15% reduction in compute credits consumed per query across diverse production workloads. ClickHouse Cloud executed this rollout using automated validation checks, scheduled maintenance windows, and capacity coordination with Google Cloud.
In large-scale ClickHouse Cloud deployments, replicas traditionally load entire primary and secondary indexes from object storage into working memory, creating massive redundant memory overhead as fleets expand. Index sharding resolves this bottleneck by distributing the index analysis phase across available replicas using consistent hashing over data parts. Instead of every node loading full index structures, each replica loads and analyzes only its assigned fraction of primary keys, bloom filters, text indexes, or vector indexes. The query initiator merges the resulting granule match ranges, allowing parallel replicas to stream data without any single machine retaining the whole index. This approach freed working memory for query execution and accelerated index analysis by up to 7.7x in benchmark tests on a 50 billion row table.
ClickHouse announced an expanded strategic collaboration with Google Cloud introducing four core architectural milestones. First, native integration with Google Cloud Lakehouse allows direct sub-second queries across data lakes and ClickHouse deployments without ETL pipelines or data duplication. Second, the Bring Your Own Cloud deployment model lets organizations run managed ClickHouse Cloud inside their own Google Cloud Virtual Private Cloud to retain data sovereignty and network controls. Third, ClickHouse Cloud is migrating automatically to Google's custom Arm-based Axion processors, boosting query throughput and cost efficiency for analytical workloads without requiring application changes. Finally, a new ClickHouse MCP server integration connects ClickHouse with Google Antigravity to enhance developer and analyst workflows.
Transactional databases handle large volumes of short, real-time read and write operations for systems such as orders, payments, account balances, and customer records. They support online transaction processing (OLTP) by treating related steps as one logical unit: successful work commits, while failures roll back the full transaction. ACID—atomicity, consistency, isolation, and durability—protects integrity under concurrency and failures, with row-oriented storage optimized for individual-record access, locking and isolation levels coordinating concurrent work, and write-ahead logging (WAL) supporting crash recovery. The source positions these databases as foundations for banking, e-commerce, healthcare, reservations, and telecommunications, while noting that row-oriented execution is less suitable for large analytical workloads and that horizontal scaling is more complex than in distributed NoSQL systems.
Stripe Fee Credits replaces ad-hoc discounts and manual cash adjustments with a programmable, auditable virtual balance dedicated to eligible Stripe fees. The architecture has three layers: a standalone gRPC credits engine for metadata, policies, lifecycle, and audit history; logical financial accounts built on Money Movement and Storage; and fee-settlement orchestration that applies eligible credits before the primary balance. Logical accounts carry ledger balances without representing real cash, while platform checks restrict Originated Money Transfers to logical-to-logical movements and exclude these accounts from cash sweeps. Prepaid and Stripe-funded credits share the user experience but use separate funds flows: prepayments create deferred revenue and collect tax at drawdown, whereas Stripe-funded credits are booked as Stripe costs with taxes written off. The resulting infrastructure supports incentives, prepaid annual fees, and other programs across 26+ countries, with Dashboard balances and real-time auditability, while avoiding bespoke engineering for each commercial model.
The best engineering reads, once a week.
