Loading…
Topic
378 posts about Scalability. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Historical market-data systems separate a durable archive from the real-time trading path, but the archive still must support research, backtesting, transaction-cost analysis, and regulatory reconstruction. The worked example uses Binance’s public archive to load trades and quotes for five USD-M perpetual-futures symbols into ClickHouse, with a MergeTree table ordered by (symbol, ts) and codecs including DoubleDelta and ZSTD. SQL queries calculate VWAP, hourly OHLC candles, and as-of joins, while ClickHouse’s sparse primary index skips most granules for symbol-and-time filters. Across ten months, 16.2 billion rows compress to 76 GiB on disk; the post reports a ClickHouse Cloud cost of $1.88 per month and argues that typical symbol-and-time queries retain low latency at that scale, while broader scans read more data.
Insider One’s Unified Customer Database (UCD) unifies event data, user attributes, identity resolution, and profiles across channels to support real-time segmentation, personalization, and analytics. The platform initially ran ClickHouse in a self-managed AWS deployment, but growth brought larger datasets, more concurrent queries, and increasing work tuning clusters and balancing performance against cost. In a proof of concept, the team migrated an anonymized dataset from one of its largest customers to ClickHouse Cloud and replayed UI segmentation, audience-count, and scheduled-campaign workloads. Average query latency fell 50–60% without significant changes, reaching 70–75% for the largest customers, while latency percentiles became more predictable; the managed service also reduced operational burden and gave the team more capacity for product development and its AI-oriented roadmap.
Large language model training workloads frequently face GPU high-bandwidth memory (HBM) capacity constraints before compute limits are reached. To alleviate this memory pressure in JAX training workloads, host offloading transfers selected forward-pass activations to pinned host memory over high-speed NVLink-C2C interconnects and reloads them during the backward pass instead of recomputing them. Benchmarks executed on 128 GPUs within NVIDIA GB200 NVL72 systems using MaxText demonstrated significant improvements. For DeepSeek-V3 671B, host offloading combined with the Latency Hiding Scheduler and pipelined offloading reached 908.2 TFLOPs/s/device, surpassing activation rematerialization throughput by 57% and unlocking larger batch configurations. Similarly, QKV activation offloading on Llama 3.1 405B achieved a 2.9% throughput increase when transfer latency was hidden behind computation.
Efficient large language model serving requires co-designing architectures to balance datacenter token throughput and user interactivity within hardware constraints. Roofline analysis shows that linear layers become memory-bound when hidden or intermediate projection dimensions are small, leaving arithmetic hardware underutilized even at large token counts. For prefill workloads, Chunked Pipeline Parallelism reduces first-token latency on long contexts when models use regular, repeatable layer patterns that split into balanced stages. For latency-critical decoding, decoupling attention and feed-forward network parallelization enables independent optimization, applying expert parallelism for feed-forward networks and Helix parallelism to shard the key-value cache across sequences. Following architectural design rules—such as dimension alignment, width prioritization, and low-precision NVFP4 compatibility—allows models to achieve higher utilization on modern accelerator hardware.
Biomolecular structure prediction and co-folding pipelines encounter latency and memory ceilings during sequence alignment and multi-residue inference. NVIDIA addresses these constraints through the BioNeMo Agent Toolkit, which connects AI agents to accelerated biological computing components. The pipeline offloads multiple sequence alignment to MMseqs2-GPU, while cuEquivariance and the OpenFold3 NIM accelerate core geometric attention kernels to cut inference latency up to fourfold. To handle large molecular assemblies beyond single-GPU memory limits, the Fold-CP framework distributes context across multiple GPUs with O(N²/P) memory scaling. Together, these tools enable the prediction of complexes up to 32,000 tokens on 64 NVIDIA B300 GPUs, making large-scale virtual screening and ribosome-scale modeling computationally feasible.
Molecular dynamics packages like GROMACS face severe scaling bottlenecks when CPU-orchestrated MPI communication introduces blocking synchronizations during halo exchanges. To resolve this, researchers implemented GPU-initiated communication using NVIDIA NVSHMEM to eliminate CPU-GPU handoffs. The redesigned approach fuses packing, remote memory access, and completion waiting into device-side kernels using fine-grained signaling. Benchmarks on NVIDIA DGX H100 systems demonstrated up to 1.5x performance gains intra-node and up to 2x on GB200 NVL72 multi-node clusters, particularly on latency-bound systems. While device-initiated communication introduces a slight 1–3% overhead on very large compute-bound configurations at low node counts, it consistently improves strong scaling for production workloads.
At Uber’s monorepo scale, traditional Git workflows create bottlenecks through multi-gigabyte clones, local checkouts, repeated synchronization, and independent fetch or push operations across automation systems. The post describes GitFarm, a Git as a Service platform exposed through a high-performance gRPC API that executes standard Git commands in secure, ephemeral sandboxes backed by pre-warmed repository and container pools. Each backend maintains a bare clone, receives push-based updates and periodic git fetch operations every five minutes, while clients can explicitly fetch when they need current state. The design provides a full checkout in under 500 milliseconds and reduced client-side resource utilization by over 80%, while avoiding local clones and long initial clone delays.
Trigger.dev, an open-source platform for serverless asynchronous tasks, needed to scale observability for increasingly complex, long-running AI workflows spanning thousands of tenants. Its initial Postgres design stored logs, traces, and events in a growing table, making indexes, analytics, and operational reliability increasingly difficult to manage. The team moved observability workloads to ClickHouse after evaluating performance, scalability, open-source compatibility, deployment flexibility, and support for dynamic telemetry, including its native JSON data type. Migration occurred gradually at the task-execution level, routing each run’s logs to either database without downtime or customer intervention. ClickHouse now supports real-time telemetry analysis, approximately 200-millisecond P95 latency for queries returning tens of thousands of rows, and a dramatically smaller telemetry storage footprint.
GPU-accelerated Presto integrates NVIDIA cuDF algorithms and NVLink to deliver low-latency analytical query execution on massive datasets. Benchmarks derived from TPC-H across scale factors 1K and 3K reveal that a single NVIDIA DGX B200 system with eight GPUs outperforms 8- to 10-node Intel Xeon clusters by up to 8.2x. Scaling out across multinode NVIDIA GB200 NVL72 systems paired with IBM Storage Scale achieved additional performance gains. Enabling NVIDIA GPUDirect Storage, tuning I/O parameters, adjusting thread counts, and applying query rebatching delivered a cumulative 64% reduction in query runtimes. Topology-aware GPUDirect Storage cold reads proved roughly twice as fast as standard POSIX reads by avoiding CPU memory staging and NUMA boundary crossing penalties.
Agentic AI workflows and reinforcement learning rely heavily on sequential CPU tasks, such as tool calls, sandboxed code execution, data processing, and environment rollouts between GPU steps. When CPU performance degrades under full socket load, training time inflates, serving latency grows, and GPU KV-caches face eviction. The NVIDIA Vera CPU addresses these bottlenecks by featuring 88 Olympus cores on a monolithic compute die, paired with power-efficient LPDDR5x memory and the Scalable Coherency Fabric. These cores incorporate a neural branch predictor, a 10-wide decode front end, deep out-of-order execution, and NVIDIA Spatial Multithreading. Consequently, Vera achieves 1.8x faster core execution, 40% lower peak loaded latency compared to x86 CPUs, and up to 1.2 TB/s total memory bandwidth, preventing costly KV-cache recomputations.
Large-scale LLM training jobs spanning thousands of GPUs frequently encounter transient hardware interruptions that bottleneck tightly coupled tensor parallel groups. Nonuniform Tensor Parallelism addresses this bottleneck by dynamically adapting the tensor parallelism degree within affected scale-up domains rather than dropping entire data replicas. When a GPU drops out, the system reconfigures the remaining functional devices to process larger shards and resumes execution from the latest checkpoint. To prevent the downsized replica from stalling global synchronization, advanced rack designs temporarily power-boost the active GPUs to increase clock frequencies and match baseline throughput. Overlapping the necessary tensor resharding with backward computation and parameter synchronization reduces overhead on healthy replicas to under one percent.
ClickHouse 26.6 is a release marking the 10-year anniversary of ClickHouse becoming open source, with 56 new features, 79 performance optimizations, and 366 bug fixes. Hypothetical skip indexes let users evaluate set indexes in a session without building them; on a 487.24-million-row UK property dataset, the example estimated skip ratios of 77.0% at granularity 128 and 92.2% at granularity 1. The release overhauls dependencies between refreshable materialized views, addressing schedule drift that could make dependent stages lag or miss a refresh cycle. It also introduces experimental continuous queries, the aiEmbed function, embedding quantization to 8 bits and below, and workload-level memory reservations.
The post examines why enterprise AI efforts can become too expensive, slow, and difficult to scale, citing survey findings that 67% of organizations with disconnected data environments identify storage, movement, and duplication as their largest recurring AI cost. It presents three infrastructure considerations: delivering infrastructure at agentic speeds, streamlining data, and adopting systems built for AI scale. Examples include instant temporary environments with secure rollback and restoration, unified operational and analytical data stored separately from compute in low-cost cloud storage, and elastic scaling that can move from high concurrency to zero in seconds. The conclusion is that open, AI-ready, purpose-built databases can reduce pipeline complexity and costs, support experimentation, and let organizations align spending with unpredictable workloads while enabling faster AI innovation.
ClickStack’s inherited OpenTelemetry Collector schema was struggling with slow searches, Top N queries, and aggregations that could time out on large observability workloads. The team prioritized common log and trace access patterns, including exact-attribute searches, body-text queries, histograms, individual log lookups, and distinct-key or value introspection. They used fixed production-like workloads and iterative testing with ClickCannon, Kubernetes-managed environments, ephemeral ClickHouse Cloud instances, and controlled schema, query, and version comparisons. The resulting redesign combined primary-key changes, text indexes, query rewrites, alias columns, materialized views, and newer ClickHouse features, reducing representative query latency by more than 5× while balancing ingestion, storage, and operational efficiency.
amazon.science
Amazon measures carbon intensity across its diverse business operations to track progress toward its goal of reaching net-zero carbon by 2040. For retail operations, the company monitors carbon emissions per unit shipped, achieving a 39 percent reduction by the end of 2025 compared to 2019 baselines. This per-unit decrease stems from targeted investments in electric delivery vehicles, carbon-free energy, smarter package routing, lightweight packaging, low-carbon fuels, and alternative transport modes like rail. To improve industry comparability and transparency, the company updated its economic carbon intensity indicator in 2025 from emissions per dollar of gross merchandise sales to emissions per dollar of revenue. Carbon intensity is also evaluated regionally and nationally to isolate geographic variations and implement localized decarbonization interventions across supply chain networks.
PgBouncer is single-threaded, so a single process can bottleneck connection pooling on multi-vCPU machines even when Postgres has capacity. ClickHouse Managed Postgres runs a fleet of PgBouncer processes sized to available cores, sharing one port through so_reuseport; peering forwards query-cancellation requests to the process that owns the session, while transaction pooling and per-process connection-budget splits limit Postgres connections. In tests on identical AWS EC2 c7i.4xlarge pooler instances, a 16-process fleet reached roughly 336,000 transactions per second at 256 clients, compared with 76,893 for one process, while using about 49% versus 8% in-guest CPU. The fleet's advantage emerged under concurrency, whereas the single process was slightly faster at eight clients, and the described setup ships by default with every ClickHouse Managed Postgres server.
LogHouse now stores 431 PiB of uncompressed data across 1.59 quadrillion rows, spans more than 30 regions across three cloud providers, and serves typical queries in a few hundred milliseconds. The platform grew 23x from 19 PiB in two years, while peak ingestion reached 80 GiB/s and 190 million rows per second. Its write architecture uses local geosharding, multiple cells in busy regions, and ClickHouse Async Inserts to combine small incoming writes into larger parts without overwhelming merges. On the read side, a single Distributed table hides the cell topology, while unavailable shards remain visible by default rather than producing silently incomplete results. Current work includes reducing index and metadata memory, improving async-insert durability, collecting system data from object storage, and expanding OpenTelemetry traces and metrics.
Clever, an identity platform for K-12 education, faced a fragmented logging setup as its 400 services and 200 AWS Lambdas generated about 150 TB of uncompressed logs monthly. Datadog indexed only about 10% and retained it for three days, while the remainder was in Athena and required SQL. Clever migrated to ClickHouse Cloud on AWS, indexing all logs with 60-day retention and reporting 200× more searchable log volume at no additional cost, with 10× compression. Its design extracts frequently queried JSON fields into columns, stores remaining attributes in a map alongside raw JSON, uses skipping indexes, date partitions, a 60-day TTL, and reverse Unix timestamp ordering; this ordering produced an 8–10× query speedup. A custom query layer translates LogQL to SQL, while separate development and production services and multi-region deployment address isolation and availability.
Jua uses ClickHouse Cloud to deliver data from EPT-2, its physics foundation model, to energy traders and grid operators faster than its previous file-based pipeline. Jua’s forecasts combine outputs from multiple models and add roughly four terabytes of new data daily, producing about one petabyte compressed and over 10–12 petabytes uncompressed. Replacing Zarr and other file-based storage plus manual pre-fetching with ClickHouse Cloud reduced flagship forecast delivery from one hour to 20 minutes, with about 20 seconds of upload overhead. The platform also cut compute costs by roughly a third, reduced two-year single-location historical queries from hours to seconds, and supports both point lookups and regional aggregations; Jua selected the managed service for operational simplicity, separated storage and compute, and cost efficiency.
Melvyn Peignon examines a Databricks keynote benchmark in which ClickHouse was shown “crashing” under workloads associated with Reyden, Databricks’ gated low-latency compute product. The post argues that the presentation omitted hardware, cost, configuration, cache settings, deployment details, and sufficient methodology for independent validation. Using the disclosed TPC-H SF1 dataset and Q6 query, ClickHouse was tested on a 30 vCPU, 120 GB node without tuning; it did not crash and sustained about 420 QPS at sub-second P90 latency. The post reports that reaching 15,000 QPS required straightforward sizing to roughly 30–40 untuned nodes, while further gains could come from settings such as thread counts, query cache, and projections. It concludes that benchmarks should be open, reproducible, and detailed enough to inspect rather than accepted as keynote claims.
The best engineering reads, once a week.
