Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Meta describes how its BLOB-storage architecture evolved to address AI workloads' storage bottlenecks, which can stall GPUs and slow research iteration as datasets and GPU deployments expand. Legacy requests traversed multiple stateful metadata layers, sometimes across regions, producing latencies of hundreds of milliseconds even when flash made millisecond access possible. The redesign consolidates metadata into a flat schema backed by ZippyDB for O(1) path resolution, removes the dataplane proxy in favor of a fat client SDK, and deploys regional BLOB-storage stacks beside GPUs. For data loading, Meta combines host memory and flash with regional flash and global HDD-backed storage as tiered caches, using ordinary and deep prefetch, on-demand hydration, and TTL or LRU eviction policies. Production rollout saw rapid adoption, while both data-loading paradigms remain supported.
PgBouncer is single-threaded, so a single process can bottleneck connection pooling on multi-vCPU machines even when Postgres has capacity. ClickHouse Managed Postgres runs a fleet of PgBouncer processes sized to available cores, sharing one port through so_reuseport; peering forwards query-cancellation requests to the process that owns the session, while transaction pooling and per-process connection-budget splits limit Postgres connections. In tests on identical AWS EC2 c7i.4xlarge pooler instances, a 16-process fleet reached roughly 336,000 transactions per second at 256 clients, compared with 76,893 for one process, while using about 49% versus 8% in-guest CPU. The fleet's advantage emerged under concurrency, whereas the single process was slightly faster at eight clients, and the described setup ships by default with every ClickHouse Managed Postgres server.
Artemis Security uses ClickHouse Cloud on AWS for real-time threat detection across many terabytes of compressed logs, tens of billions of daily rows, and queries over trillions of rows. To reduce repeated scans, its engineers coalesce hundreds of detection-rule predicates into one query, tagging matching rows by rule ID; batches of more than 100 rules now run in under a second, with a measured 69x speedup. A Claude Code skill called /poke connects ClickHouse system tables, CloudWatch, Datadog, application data, and source code to investigate infrastructure issues through natural-language prompts. For investigative queries over semi-structured JSON, Artemis extracts frequently queried fields into a parallel materialized table and uses a hybrid two-step lookup, producing 30–60x faster queries and up to 400x lower CPU consumption.
GPU-accelerated query engines encounter performance constraints stemming from memory and I/O bandwidth limitations. NVIDIA introduces GQE, a reference architecture using NVIDIA cuDF, nvCOMP, and other CUDA-X libraries to accelerate SQL query execution across modern hardware like the NVIDIA GB200 NVL4. The architecture coordinates three distinct layers: a query layer consuming Substrait plans, a data layer managing in-memory table transfers, and an execution layer executing relational operator task graphs. GQE employs pipeline parallelism, partition pruning, hybrid compression, and batched asynchronous transfers to overlap data movement with GPU compute. On a non-standard 1 TB TPC-H benchmark, GQE on a single B200 GPU achieved a 7.5x total execution time speedup over dual-socket CPU configurations running DuckDB.
NVIDIA Omniverse NuRec reconstructs dynamic 3D environments from camera and lidar sensor data for autonomous vehicle simulation workflows, but high computational demands historically caused long reconstruction times. Engineers used NVIDIA Nsight Systems and Nsight Compute to systematically profile the PyTorch-based training pipeline and optimize its underlying CUDA kernels. Nsight Systems revealed GPU underutilization caused by numerous small kernels and blocking synchronization calls, leading to a fused interpolate kernel that accelerated execution from 4.184 milliseconds to 83.81 microseconds. Profiling with Nsight Compute uncovered resource mismatch in the renderBackward kernel, prompting the team to split it into dedicated camera and lidar implementations with tuned register and shared memory allocations. These adjustments raised kernel occupancy from approximately 15% to between 30% and 50% while decreasing the longest lidar kernel runtime from 31 milliseconds to 18 milliseconds.
LogHouse now stores 431 PiB of uncompressed data across 1.59 quadrillion rows, spans more than 30 regions across three cloud providers, and serves typical queries in a few hundred milliseconds. The platform grew 23x from 19 PiB in two years, while peak ingestion reached 80 GiB/s and 190 million rows per second. Its write architecture uses local geosharding, multiple cells in busy regions, and ClickHouse Async Inserts to combine small incoming writes into larger parts without overwhelming merges. On the read side, a single Distributed table hides the cell topology, while unavailable shards remain visible by default rather than producing silently incomplete results. Current work includes reducing index and metadata memory, improving async-insert durability, collecting system data from object storage, and expanding OpenTelemetry traces and metrics.
Clever, an identity platform for K-12 education, faced a fragmented logging setup as its 400 services and 200 AWS Lambdas generated about 150 TB of uncompressed logs monthly. Datadog indexed only about 10% and retained it for three days, while the remainder was in Athena and required SQL. Clever migrated to ClickHouse Cloud on AWS, indexing all logs with 60-day retention and reporting 200× more searchable log volume at no additional cost, with 10× compression. Its design extracts frequently queried JSON fields into columns, stores remaining attributes in a map alongside raw JSON, uses skipping indexes, date partitions, a 60-day TTL, and reverse Unix timestamp ordering; this ordering produced an 8–10× query speedup. A custom query layer translates LogQL to SQL, while separate development and production services and multi-region deployment address isolation and availability.
Jua uses ClickHouse Cloud to deliver data from EPT-2, its physics foundation model, to energy traders and grid operators faster than its previous file-based pipeline. Jua’s forecasts combine outputs from multiple models and add roughly four terabytes of new data daily, producing about one petabyte compressed and over 10–12 petabytes uncompressed. Replacing Zarr and other file-based storage plus manual pre-fetching with ClickHouse Cloud reduced flagship forecast delivery from one hour to 20 minutes, with about 20 seconds of upload overhead. The platform also cut compute costs by roughly a third, reduced two-year single-location historical queries from hours to seconds, and supports both point lookups and regional aggregations; Jua selected the managed service for operational simplicity, separated storage and compute, and cost efficiency.
Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.
Next.js provides out-of-the-box tracing for incoming requests, fetch calls, middleware, and server-side rendering, but traces remain invisible without a configured exporter. Developers can use the @vercel/otel package inside an instrumentation file to initialize the OpenTelemetry SDK and transmit traces to any OTLP-compatible destination. While Next.js automatically creates spans for standard request lifecycles, developers can define custom active spans in the Node runtime to capture specific domain operations and contextual attributes. Exporting traces to backends like Sentry requires configuring standard OTLP endpoint and authentication environment variables. Choosing between direct OTLP export via @vercel/otel and the dedicated Sentry SDK depends on requirements, as direct export lacks browser tracing, error monitoring, and Edge custom spans.
Melvyn Peignon examines a Databricks keynote benchmark in which ClickHouse was shown “crashing” under workloads associated with Reyden, Databricks’ gated low-latency compute product. The post argues that the presentation omitted hardware, cost, configuration, cache settings, deployment details, and sufficient methodology for independent validation. Using the disclosed TPC-H SF1 dataset and Q6 query, ClickHouse was tested on a 30 vCPU, 120 GB node without tuning; it did not crash and sustained about 420 QPS at sub-second P90 latency. The post reports that reaching 15,000 QPS required straightforward sizing to roughly 30–40 untuned nodes, while further gains could come from settings such as thread counts, query cache, and projections. It concludes that benchmarks should be open, reproducible, and detailed enough to inspect rather than accepted as keynote claims.
Moving large model weights efficiently requires quantization, but mapping weights to NVFP4's eight positive values risks severe precision loss or outlier-induced underflow. To address this, NVIDIA quantized the 550B Nemotron 3 Ultra model using NVIDIA Model Optimizer with a multi-precision strategy rather than quantizing all layers uniformly. The recipe applies four-over-six adaptive block scaling to Mixture of Experts routed weights, while assigning FP8 to shared experts and retaining BF16 for sensitive attention linears and embeddings. This process reduced the checkpoint size from 1,121 GB down to 352.3 GB and enabled single-checkpoint execution across Hopper using W4A16 and Blackwell using native W4A4. As a result, the model achieves up to 5.9x higher inference throughput than the GLM-5.1 754B FP4 model on decode-heavy workloads while matching BF16 benchmark accuracy.
ClickHouse has achieved the AWS Cloud Operations Competency in the Monitoring and Observability category, recognizing validated expertise in operating reliable AWS systems with logs, metrics, and traces. The post describes observability as a high-volume, high-cardinality, high-concurrency workload where legacy systems may require sampling, field reduction, or short retention. ClickHouse Cloud uses columnar storage, compression, parallel queries, compute-compute separation, and ClickPipes integrations with Amazon Kinesis, Amazon MSK, and Amazon S3 to support full-fidelity telemetry. Customer examples include Modal’s 1 to 2 million events per minute, Exabeam’s 1.2 million events per second per region, and Qonto’s reduction of 231 TB to 376 GB. The company also describes MCP servers, AI Notebooks, and managed ClickStack as investments for collaborative incident investigation.
Vulkan introduces the VK_EXT_descriptor_heap extension to refactor GPU resource binding, replacing set-based layouts and pool allocations with application-managed descriptor memory heaps. This approach aligns Vulkan closer to Direct3D 12 and benefits workloads like dynamic texture indexing and complex ray tracing shaders. Developers manage memory directly, binding a single resource heap and a separate sampler heap globally to avoid high rebinding overhead. Shaders can access descriptors either through mapping structures using constant offsets or push constants, or directly via untyped pointers enabled by VK_KHR_shader_untyped_pointers. Support is integrated into NVIDIA drivers 610, Vulkan Headers 1.4.340, and Nsight Graphics 2026.2 for frame capture and inspection.
Silk is a C++ stackful-fiber library and scheduler created for ClickHouse’s highly concurrent, I/O-bound components, including object-storage I/O, distributed cache lookups, replica coordination, and HTTP fan-out. Its design combines one pinned OS scheduler thread per CPU, NUMA-aware work stealing, io_uring for asynchronous I/O and timers, and zero heap allocation in the steady-state hot path; fiber stacks are mmap’d from per-fiber pools with guard pages, while runtime containers are intrusive. The post reports benchmarks including roughly 3.6-nanosecond fiber yields with cross-CPU stealing, 5.9 million file IOPS in a working configuration, and about fifteen times boost::asio’s throughput at one connection. At 10,000 concurrent S3-style requests, internal measurements showed a roughly 65% improvement in 99.9th-percentile latency versus an equivalent thread-pool executor, with identical median throughput; distributed cache is the first intended ClickHouse integration.
Generative AI media pipelines can exceed the memory and compute capacity of a single GPU, creating a deployment challenge for models that retain TensorRT’s kernel fusions, memory planning, and quantization. TensorRT 11.0 introduces native multi-device inference and IDistCollectiveLayer primitives, while NCCL supplies topology-aware transport across NVLink, NVSwitch, PCIe, and InfiniBand for supported distributed collectives. The article contrasts tensor parallelism, which shards layer weights, with context parallelism, which partitions sequences; it then focuses on AllGather KV, Ring Attention, and DeepSpeed Ulysses for long-sequence media attention. It also describes converting PyTorch pipelines with Torch-TensorRT into C++-deployable engines, benchmarking Cosmos 3 and FLUX.1 on an eight-GPU node, and provides C++ and OpenMPI steps for configuring an eight-rank NCCL deployment.
KRAFTON developed PUBG Ally, an on-device AI co-playable character for PUBG: BATTLEGROUNDS, using NVIDIA ACE tooling and models. Conventional non-playable characters rely on fixed dialogue, but PUBG Ally processes live game states and player voice input in real time. The interactive pipeline combines automatic speech recognition, a quantized Mistral-NeMo-Minitron-2B small language model, and a custom text-to-speech model running locally on client GPUs with at least 8GB of VRAM. To minimize latency, the architecture separates reactive gameplay into a behavior-tree System 1 layer while routing strategic decisions and speech generation to the language model. KRAFTON tuned the non-deterministic system across multi-layer evaluations and large-scale playtests with over a thousand players.
Pinterest’s foundation ranking models store approximately 99% of their parameters in embedding tables, causing cross-node NCCL all-to-all communication to severely bottleneck multi-node distributed training. Initial multi-node scaling stalled at 1.21x on four nodes even after adopting AWS Elastic Fabric Adapter for OS-bypass networking. To resolve this, engineers profiled GPU activity with PyTorch Profiler and systematically addressed communication bottlenecks across multiple layers. The team implemented FP8 quantized communications using FBGEMM, balanced table sharding across GPUs, halved embedding dimensions while doubling row counts, and adopted a 2D parallel communication topology alongside PyTorch Distributed Checkpoint. These combined interventions achieved near-linear scaling factors of 3.9x on four nodes and 7.5x on eight nodes across 64 GPUs.
ClickHouse Cloud built WAL-RUS, an open-source Rust implementation of PostgreSQL backup and WAL archival tooling, after encountering difficulty predicting WAL-G’s memory usage in resource-constrained environments. The project preserves WAL-G’s core functionality and compatibility while using bounded worker pools, controlled concurrency, persistent object-storage connections, and a streaming pipeline that limits buffering and data copies. It uses the same WALG_ configuration variables, and WAL-G and WAL-RUS can read each other’s archives. In a sustained WAL-heavy benchmark with four archival workers, WAL-RUS stayed below 1 GB of peak virtual memory versus nearly 2.8 GB for WAL-G, while both maintained minimal backlog and comparable CPU usage. The authors plan to make WAL-RUS the default backup and archival mechanism for ClickHouse Cloud’s managed Postgres offering and are working to upstream its Postgres 17 wal summaries support to WAL-G.
Vibe.co, recently acquired by Walmart, uses ClickHouse Cloud to provide real-time campaign reporting for thousands of Connected TV advertisers across billions of ad impressions. It migrated from Postgres after aggregation-heavy OLAP queries made its pre-aggregation layer a bottleneck, limiting interactive slicing and requiring new jobs for reporting use cases. The team evaluated self-hosted ClickHouse, another managed option, and ClickHouse Cloud, choosing the latter for operational simplicity, support from ClickHouse engineers, and separation of storage and compute with more predictable costs. Vibe uses refreshable materialized views for hot/cold tiering, keeping the last 30 days in a hot table; stored data grew from about 100 GB to over 2 TB without rearchitecting, while more than 90% of client campaign reports respond in under 100ms.
The best engineering reads, once a week.
