Loading…
Topic
378 posts about Scalability. Every summary links to the original.
See the idea
Consolidating agent memory, retrieval, and governance onto MongoDB Atlas removes sync and vendor layers, but ties scaling and control to one platform. This trade-off view shows the payoff, the cost, and the condition that decides it.
Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.
Apache Pinot is a distributed real-time OLAP datastore for low-latency analytics. This architecture view shows how streams, processing, partitioning, indexed storage, and replica-aware serving fit together.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
ClickHouse has achieved the AWS Cloud Operations Competency in the Monitoring and Observability category, recognizing validated expertise in operating reliable AWS systems with logs, metrics, and traces. The post describes observability as a high-volume, high-cardinality, high-concurrency workload where legacy systems may require sampling, field reduction, or short retention. ClickHouse Cloud uses columnar storage, compression, parallel queries, compute-compute separation, and ClickPipes integrations with Amazon Kinesis, Amazon MSK, and Amazon S3 to support full-fidelity telemetry. Customer examples include Modal’s 1 to 2 million events per minute, Exabeam’s 1.2 million events per second per region, and Qonto’s reduction of 231 TB to 376 GB. The company also describes MCP servers, AI Notebooks, and managed ClickStack as investments for collaborative incident investigation.
Silk is a C++ stackful-fiber library and scheduler created for ClickHouse’s highly concurrent, I/O-bound components, including object-storage I/O, distributed cache lookups, replica coordination, and HTTP fan-out. Its design combines one pinned OS scheduler thread per CPU, NUMA-aware work stealing, io_uring for asynchronous I/O and timers, and zero heap allocation in the steady-state hot path; fiber stacks are mmap’d from per-fiber pools with guard pages, while runtime containers are intrusive. The post reports benchmarks including roughly 3.6-nanosecond fiber yields with cross-CPU stealing, 5.9 million file IOPS in a working configuration, and about fifteen times boost::asio’s throughput at one connection. At 10,000 concurrent S3-style requests, internal measurements showed a roughly 65% improvement in 99.9th-percentile latency versus an equivalent thread-pool executor, with identical median throughput; distributed cache is the first intended ClickHouse integration.
Launched in 2016, MongoDB Atlas was designed to bring MongoDB's developer experience to the cloud while eliminating operational overhead for production environments. Over ten years, Atlas expanded beyond a managed database into a multi-cloud data platform that supports multi-document ACID transactions, Queryable Encryption, native search, and stream processing. The platform introduced Atlas Vector Search in 2023 and acquired Voyage AI in 2025 to embed advanced embedding and reranking models natively. These integrated capabilities address the requirements of retrieval-augmented generation and autonomous AI agents without relying on disparate vector stores or sync pipelines. Atlas now serves more than three trillion queries daily across over 125 AWS, Google Cloud, and Microsoft Azure regions for nearly 70,000 customers.
Pinterest’s foundation ranking models store approximately 99% of their parameters in embedding tables, causing cross-node NCCL all-to-all communication to severely bottleneck multi-node distributed training. Initial multi-node scaling stalled at 1.21x on four nodes even after adopting AWS Elastic Fabric Adapter for OS-bypass networking. To resolve this, engineers profiled GPU activity with PyTorch Profiler and systematically addressed communication bottlenecks across multiple layers. The team implemented FP8 quantized communications using FBGEMM, balanced table sharding across GPUs, halved embedding dimensions while doubling row counts, and adopted a 2D parallel communication topology alongside PyTorch Distributed Checkpoint. These combined interventions achieved near-linear scaling factors of 3.9x on four nodes and 7.5x on eight nodes across 64 GPUs.
Vibe.co, recently acquired by Walmart, uses ClickHouse Cloud to provide real-time campaign reporting for thousands of Connected TV advertisers across billions of ad impressions. It migrated from Postgres after aggregation-heavy OLAP queries made its pre-aggregation layer a bottleneck, limiting interactive slicing and requiring new jobs for reporting use cases. The team evaluated self-hosted ClickHouse, another managed option, and ClickHouse Cloud, choosing the latter for operational simplicity, support from ClickHouse engineers, and separation of storage and compute with more predictable costs. Vibe uses refreshable materialized views for hot/cold tiering, keeping the last 30 days in a hot table; stored data grew from about 100 GB to over 2 TB without rearchitecting, while more than 90% of client campaign reports respond in under 100ms.
amazon.science
Amazon is investing in next-generation nuclear technology powered by tristructural isotropic (TRISO) fuel particles to satisfy the increasing energy demands of cloud computing and artificial intelligence infrastructure. Each sub-millimeter TRISO particle encloses a high-assay low-enriched uranium kernel within a porous carbon buffer and a three-layer ceramic shell featuring silicon carbide that acts as a robust miniature containment vessel. Testing by the U.S. Department of Energy demonstrated that irradiated particles withstand temperatures of 1600°C for 300 hours without detectable failures. In pebble-bed designs like the X-energy Xe-100 reactor, graphite spheres containing TRISO fuel circulate continuously through the core to enable online refueling, passive heat removal, and modular 80-megawatt deployments at facilities like the Cascade Energy Center.
Power costs comprise up to 40% of AI factory operating expenses, making performance per watt critical within fixed site power limits. Operators maximize throughput by combining architectural enhancements, narrow-precision formats like NVFP4, and mixture-of-experts model designs that activate fewer parameters per token. During distributed LLM training in Megatron-LM, tuning individual GPU speeds along critical execution paths eliminates idle power waste without extending total training time. Additionally, the NVIDIA DSX platform orchestrates dynamic power allocation, 45°C direct-to-chip liquid cooling, and grid integration across racks and facilities. Together, these full-stack optimizations enable performance-optimized AI factories to deliver up to 2.6 times more tokens per second per megawatt.
Telecom operators currently operate mostly within Levels 2 and 3 of the TM Forum autonomous networks taxonomy, relying on predefined scripts. Progressing to Levels 4 and 5 demands autonomous systems capable of intent comprehension, real-time telemetry sensing, and cross-domain action coordination. To address this, an autonomy platform architecture combines domain-specific reasoning models, agent harnesses, and isolated sandboxed runtimes. Workflows separate responsibilities among on-demand agents, long-running execution loops, and deep-research multi-agent systems that evaluate ranked remediation paths or design wireless algorithms. Initial experiments in simulated SR-MPLS routing and wireless physical or MAC layer design show autonomous discovery can deliver measurable gains over standard operational and engineering baselines.
Netflix transitioned its managed batch compute infrastructure from a homegrown solution called Compute Managed Batch to Kueue on its Titus container platform. CMB previously relied on custom scheduling and admission-only fair sharing without preemption, making feature development cumbersome as the Kubernetes ecosystem evolved. To modernize the platform transparently, Netflix mapped internal tenants to Cohorts and leaf tenants to ClusterQueues and LocalQueues while routing jobs through a custom Kueue router. Kueue operates alongside existing Titus scheduling profiles rather than replacing the kube-scheduler, preserving cluster placement efficiency. The migration was completed in four weeks across millions of batch workloads, significantly increasing average resource utilization through preemption-based fair sharing.
Stripe describes integrating stablecoin settlement into its existing Global Payments and Treasury Network (GPTN), rather than creating a parallel system. GPTN models accounts, wallets, and payment rails as a directed graph, allowing routing, netting, cross-entity settlement, FX, reconciliation, and liquidity management to span fiat and blockchain flows. For a payout, Stripe records a pending stablecoin obligation, Bridge draws from a stablecoin liquidity pool to credit the recipient within seconds to minutes, and fiat replenishment proceeds asynchronously over traditional rails; reverse flows work similarly. A direct mode bypasses the usual few-minute planning cycle for urgent wallet funding, trading netting efficiency for speed, while standard flows continue batching. The live infrastructure handles real traffic, with ongoing work focused on forecast-driven liquidity, faster replenishment through RTP and SEPA Instant, and broader product and corridor coverage.
Real-time analytics is presented as a foundational workload for customer-facing analytics, observability, fraud detection, operational applications, and AI agents rather than a niche use case. The discussion argues that performance claims require open, transparent, reproducible benchmarks that disclose methodology, datasets, queries, results, and operating conditions, including continuous ingestion, concurrency, memory pressure, and fresh data. A real-time database should support queryable data within milliseconds to a couple of seconds, incremental transformations, bounded tail latency as writes increase, configurable read-time versus write-time work, unified support for text, vector, JSON, and structured analytics, and efficiency as workload scales. The article connects renewed attention to agentic workloads, whose continuous, high-volume querying makes low latency, concurrency, and fresh context increasingly necessary, and calls for shared standards beyond a single latency number.
Appcues replaced an analytics stack that included Snowflake and Airflow after rising scale pushed queries to 20 seconds or more and limited reports to 10-minute windows. The company evaluated ClickHouse Cloud on AWS for real-time segmentation, exporting Snowflake data to Parquet in S3 and testing complex email segments with nested conditions, historical events, and negative logic. Production events now pass through Kafka and a clickwriter consumer, while a clickclient library built on Elixir’s Ecto generates controlled ClickHouse queries; SharedReplacingMergeTree, table engines, and materialized views were tuned during a one-year migration. Across 1.31 PB and 410 billion events, P95 queries fell below two seconds, ingestion latency dropped to about five seconds, and analytics costs decreased 23%, while new funnel reports and AI-powered insights were added.
GLM-5.2 is presented as a flagship model for long-horizon tasks, extending GLM-5.1’s capability to a solid 1M-token context for sustained engineering work. It was trained extensively on 1M-context coding-agent scenarios involving large-scale implementation, automated research, performance optimization, and complex debugging, and adds effort-level controls to balance capability, latency, and computational cost. Its architecture uses IndexShare, sharing one lightweight indexer across every four transformer layers and reducing per-token FLOPs by 2.9× at 1M context; MTP changes for speculative decoding increase acceptance length by up to 20%. On reported long-horizon and standard coding benchmarks, GLM-5.2 is described as the highest-ranked open-source model: it trails Opus 4.8 by 1% on FrontierSWE, ranks second only to Opus 4.8 on PostTrainBench, and scores 81.0 on Terminal-Bench 2.1 versus GLM-5.1’s 63.5.
Shopify’s Catalog must reconcile billions of listings whose merchants use different product and variant structures, so AI shopping agents can identify the same real-world product. Product clustering groups related listings under a Universal Product Identifier, with precision prioritized over recall because wrong products can lead to incorrect purchases while missed variants remain findable. The intra-store system starts with rules-based singleton detection, then uses ANN and sparse average linkage for pre-chunking, and applies a two-stage LLM process that proposes clusters and critiques them using the core value proposition framework. Dynamic per-chunk strict JSON schemas enforce complete product coverage, support arbitrary chunking, and improved recall by 8% on the Toloka evaluation dataset in one schema change. The resulting pipeline is presented as a scalable intra-store foundation, while cross-store clustering into global UPIs is identified as the next step.
NVIDIA submitted results across all MLPerf Training v6.0 benchmarks, setting performance records on workloads including the DeepSeek-V3 and GPT-OSS-20B Mixture of Experts models. To scale training up to 8,192 Blackwell GPUs, NVIDIA combined hardware cluster designs with scale-out networking via Spectrum-X Ethernet and Quantum InfiniBand. The engineering team eliminated CPU-GPU synchronization bottlenecks in token-dropless MoEs by implementing full-iteration CUDA graphs with synchronization-free operators and paged stashing. Further software optimizations included CuTe DSL kernel fusions, an MXFP8 attention block in the Transformer Engine, and transitioning router elementwise math to FP32. Across these benchmarks, the GB300 NVL72 platform achieved the fastest time to train at scale and the highest normalized per-accelerator performance.
Databricks introduces Lakehouse//RT, a real-time data warehouse designed for operational analytics, BI, app serving, and observability workloads. It is powered by Reyden and is intended to deliver millisecond performance directly on lakehouse data without copying it into a separate serving layer. Preview participants saw up to 16x better performance, with response times as low as 10ms on smaller datasets, sub-100ms on larger ones, and sub-100ms latency at 12,000 queries per second on standard analytical benchmarks. Tests covering concurrency, dataset scale, and complex TPCDS queries report low latency where alternatives slowed or failed. Lakehouse//RT is in Beta for select read-only workloads, with incremental autoscaling and automatic baseline compute sizing.
ClickHouse’s tenth-anniversary history describes its path from an internal web-analytics experiment to an open-source analytical database with more than 2,000 contributors. The original problem was processing rapidly growing, real-time logs while answering flexible reports quickly; tested MySQL extensions and standalone analytical databases did not meet the stated scale of 100 billion records per day and 500 columns. The project built column-oriented storage, lightweight compression, custom aggregation structures, background merges, and query support for grouping, aggregation, filtering, and sorting. Released publicly on June 15, 2016, ClickHouse is presented as a level-three open-source project with public contribution guidelines, code review, testing, CI, documentation, and a roadmap, while contributors receive credit even when implementations are substantially rewritten.
AI development teams face constant shifts in model providers and agent frameworks, demanding data platforms that provide scalable, real-time context management. Agentic workloads require blending unstructured enterprise data, short-term session state, and persistent long-term memory. MongoDB addresses these requirements through its native JSON document model and integrated retrieval capabilities, combining full-text search, vector search, and hybrid search directly over operational data. Customer implementations such as DevRev, ElevenLabs, and Adobe rely on Atlas to achieve sub-100 millisecond hybrid retrieval and handle billions of requests. Additionally, MongoDB is collaborating with LangChain and ecosystem partners to establish open reference architectures and shared interfaces for portable agent memory across frameworks.
ClickCannon was created to answer how much CPU, memory, and disk are needed for ClickStack observability workloads, while accounting for schema, ingestion rate, hardware, and concurrent queries. It replays about 2 TiB of real OpenTelemetry trace data, converted to ClickHouse Native format with ZSTD compression, rather than relying only on synthetic datasets. A custom multithreaded Go reader/inserter provides control over uncompressed throughput and exposes stream metrics such as compressed and uncompressed bytes, row counts, and block counts, while the framework simulates users and query workloads during ingestion. The project evolved into an open-source, generic ClickHouse benchmarking tool for custom insert and query workloads, with fine-grained control over throughput, concurrency, and user behavior; the post reports that it can push substantial throughput from a single machine.
ClickHouse and Datadog announce a partnership aimed at reducing the tradeoff between retaining complete telemetry and keeping it accessible through familiar observability workflows. New Preview capabilities let organizations route logs directly to ClickHouse through Datadog Observability Pipelines and search those logs from the Datadog Log Explorer. Pipelines can parse, enrich, filter, transform, and redact data before storage, while ClickHouse Cloud’s separation of storage and compute supports long-term retention, independently scaled query resources, and workload isolation. Federated log search queries data directly in ClickHouse, avoiding duplicate copies or re-ingestion while allowing high-volume logs to remain searchable and historical investigations to use full-fidelity telemetry. The partnership combines Datadog’s collection and investigation experience with ClickHouse’s storage and analytics foundation; the native integration and federated search are available in Preview.
The best engineering reads, once a week.
