Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Bird's-eye-view (BEV) pooling frequently becomes an inference latency bottleneck in spatial AI pipelines due to irregular memory access patterns and scatter-reduce operations. BEVPoolV3 addresses this by tailoring kernel execution to the GPU memory regime, specifically distinguishing between DRAM-bound and L2-cache-resident working sets. The implementation reduces redundant scatter traffic via depth-outer traversal, precomputes indices to eliminate runtime integer division, minimizes duplicate depth loads, and adopts interval-owned writes to bypass atomic operations. On a canonical 49 MB nuScenes working set, BEVPoolV3 accelerates TensorRT plugin execution from 274.0 µs down to 16.4 µs on an NVIDIA RTX PRO 6000 Blackwell Max-Q GPU. Furthermore, profiling with Nsight Compute reveals that L2-resident scatter-reduce operators achieve optimal efficiency with FP8, avoiding the ALU instruction overhead of NVFP4 decoding.
Visa used ClickHouse Cloud to modernize reporting on Authorize.net payments data, replacing multi-day analyst-led turnaround with natural-language queries for internal teams. The solution pairs LibreChat, an open-source conversational interface, with the ClickHouse MCP server and runs on ClickHouse Cloud on AWS, allowing agents to explore and query a roughly 40 PB data footprint containing millions of rows. ClickHouse materialized views pre-compute and incrementally update results, reducing scanned data and bringing example analyses down to 100–200 milliseconds, with reported speedups of roughly 30–70 times. The platform also uses encrypted transfers, Amazon S3, VPC PrivateLink, change data capture, and federated access controls, while Visa reports identifying millions of dollars in revenue risk and reclaiming 8–10 hours per user each week.
Managed ClickStack in ClickHouse Cloud now includes the ClickStack MCP server, extending its observability investigation capabilities to external agents and AI workflows. Unlike the generic ClickHouse MCP server’s SQL-oriented interface, it offers semantic tools for log patterns, outlier correlation, slow traces, and repeatable investigations across logs, metrics, and traces. For event patterns, the server samples matching events, processes them with Drain3 in the MCP layer, and extrapolates and ranks counts, reducing output cardinality and making query overhead fixed rather than linear in the full dataset. Internal benchmarks report 25% fewer tool calls, 2.5x greater consistency, and an almost 20% increase in evaluation scores versus the standard ClickHouse MCP. Managed deployments also support full-fidelity telemetry, dedicated compute for agent workloads, and orchestration such as persisted dashboards, while native SQL remains available for unusual or deeper questions.
Power costs comprise up to 40% of AI factory operating expenses, making performance per watt critical within fixed site power limits. Operators maximize throughput by combining architectural enhancements, narrow-precision formats like NVFP4, and mixture-of-experts model designs that activate fewer parameters per token. During distributed LLM training in Megatron-LM, tuning individual GPU speeds along critical execution paths eliminates idle power waste without extending total training time. Additionally, the NVIDIA DSX platform orchestrates dynamic power allocation, 45°C direct-to-chip liquid cooling, and grid integration across racks and facilities. Together, these full-stack optimizations enable performance-optimized AI factories to deliver up to 2.6 times more tokens per second per megawatt.
Sequential token generation in autoregressive LLMs limits GPU utilization and serving throughput, while conventional speculative decoding remains constrained by autoregressive draft models. DFlash addresses this bottleneck by employing a lightweight block-diffusion drafter that predicts an entire block of masked candidate tokens in a single forward pass. The architecture incorporates target hidden-state conditioning and key-value injection into draft model projections while relying on the target model for parallel verification. Evaluated on NVIDIA Blackwell GPUs across TensorRT-LLM, vLLM, and SGLang, DFlash increases gpt-oss-120b throughput by more than 15x at high interactivity targets and delivers up to 5.8x speedups on Gemma 4 31B compared to autoregressive decoding.
pg_clickhouse v0.3.2 updates the Postgres interface for querying ClickHouse, with changes centered on PostgreSQL 19 Beta1 support, TLS configuration, regular-expression pushdown, and memory consumption. The release adds CREATE SERVER options for forcing TLS or plaintext, selecting a minimum TLS version, and enabling ClickHouse native-protocol compression. Revised pushdown logic maps Postgres regular-expression flags to ClickHouse behavior, while regexp_match() can now be pushed down when its pattern has no capturing groups. Two memory issues were fixed: one involving unbuffered HTTP-driver queries and another involving repeated foreign scans such as parameterized inner scans in nested-loop joins. The minor update requires no reload, restart, or ALTER EXTENSION UPDATE, and also fixes empty-array ANY() errors on ClickHouse versions before 25.
ParallelKernelBench (PKB) measures whether language models can replace PyTorch + NCCL implementations with fast CUDA kernels for multi-GPU workloads, where communication can account for over 20% of inference latency. Its 87 problems come from real codebases and span tensor, context, data, expert, sequence, and FSDP/ZeRO parallelism, with hardware topology supplied and symmetric memory used for direct GPU communication over NVLink. In zero-shot evaluation, GPT-5.5 solved 28 problems and produced 22 faster-than-baseline solutions; sampling three attempts raised those figures to 36 and 27, but the best fast 1@3 result was only 31%. Models performed best on familiar collective primitives, tensor-parallel GEMMs, and Ulysses-style context parallelism, while failures often involved rank coordination, data partitioning, collective ordering, and underuse of TMA or NVLS.
Transformers.js browser apps currently redownload and recache identical AI model and WebAssembly resources when they are requested from different origins. Chrome's cache uses a Network Isolation Key composed of the top-level site and current-frame site, so matching CDN URLs do not produce cache hits; the toy example incurs 177 MB of duplicate model storage, while a shared ONNX Runtime Wasm file is 4,733 kB. The proposed Cross-Origin Storage (COS) API addresses this by identifying large files with cryptographic hashes and sharing them across opted-in origins, with a polyfill extension enabling experimentation before native browser support. In the extended Transformers.js flow, a COS lookup can serve resources in milliseconds, while a miss downloads and stores them, and ModelRegistry.is_pipeline_cached() can help select among models already cached.
CostBench compares Snowflake and ClickHouse Cloud as end-to-end real-time analytics systems rather than measuring reads on a static, prepared dataset. It evaluates continuous ingest, raw-data organization, pre-aggregation freshness, and continuous query serving while background work continues, testing Snowflake’s standard-table and Interactive Tables paths alongside ClickHouse. The benchmark is designed around fresh data becoming query-ready as it arrives and distinguishes sustained real-time ingest from bulk loading or backfill. In the reported results, ClickHouse achieved 969× better cost-performance than Snowflake’s standard materialized-view setup and 180× better than its Interactive Tables setup; among the tested paths, only ClickHouse incremental materialized views kept pre-aggregations continuously synchronized without lag.
NVIDIA CCCL runtime introduces modern C++ abstractions for fundamental CUDA programming concepts starting in CUDA Toolkit 13.2 and CCCL 3.2. Designed as an alternative to the traditional CUDA runtime API, the library eliminates implicit global state by requiring explicit dependencies, such as binding streams directly to specific device references. The API enforces strong typing through owning types and non-owning reference types, omits default stream usage to ensure all streams remain non-blocking, and handles errors via standard C++ exceptions. In addition, CCCL runtime simplifies kernel launches through kernel functors that enable automatic template argument deduction and automatically transforms owning buffer arguments into device spans.
Scientific instruments and high-bandwidth sensors generate immense data volumes, but traditional architectures rely on a store-then-analyze model that bottlenecks real-time discovery and forces extensive data discarding. NVIDIA DAQIRI shifts data acquisition to a software-centric pipeline that connects streaming detectors directly to GPU computing. Built as part of the NVIDIA Holoscan Platform, DAQIRI uses DPDK to bypass the Linux kernel and move UDP and RoCE v2 traffic directly from the NIC into GPU DMA buffers at line rates exceeding hundreds of gigabits per second. The system automates packet reordering, flow steering, and on-GPU data conversions such as mapping wire int4 payloads into fp16 tensors using YAML configuration files and simple C++ or Python APIs. This enables real-time edge processing, filtering, and model inference across experimental pipelines like CERN ATLAS.
PP-OCRv6 is PaddleOCR’s latest universal OCR model family, targeting text detection and recognition across documents, screenshots, displays, labels, and scene text. It offers tiny, small, and medium tiers spanning 1.5M to 34.5M parameters, while the small and medium tiers support 50 languages. Its PPLCNetV4 backbone is shared across detection and recognition, with RepLKFPN for lightweight multi-scale detection and EncoderWithLightSVTR for recognition using local context and global attention. On PaddleOCR’s official in-house multi-scenario benchmarks, PP-OCRv6_medium reaches 86.2% detection Hmean and 83.2% recognition accuracy, improving over PP-OCRv5_server by 4.6 percentage points and 5.1 percentage points. PaddleOCR provides Paddle Inference, Transformers, and ONNX Runtime paths, and results can be saved as visualization images or structured JSON for downstream workflows.
OpenClaw receives hundreds of issues and pull requests daily, creating a triage problem for maintainers who need near-real-time filtering without consuming a paid closed-model quota. The system uses Gemma and Qwen local models inside a Pi-based agent harness: each item is normalized, enriched with GitHub context, classified into finite topic labels, and required to return structured output. For pull requests, the agent can inspect the repository through reposhell, a read-only command-restricted shell, while localpager, SQLite, a worker queue, and deterministic notification policy handle orchestration and Discord delivery. Tests described for gemma-4-26b-a4b and qwen3.6-35b-a3b report hundreds of tokens per second and good zero-shot classification accuracy without fine-tuning, while an LLM audit loop monitors errors; the authors present the pattern as agentic classification for high-throughput triage.
Stripe describes integrating stablecoin settlement into its existing Global Payments and Treasury Network (GPTN), rather than creating a parallel system. GPTN models accounts, wallets, and payment rails as a directed graph, allowing routing, netting, cross-entity settlement, FX, reconciliation, and liquidity management to span fiat and blockchain flows. For a payout, Stripe records a pending stablecoin obligation, Bridge draws from a stablecoin liquidity pool to credit the recipient within seconds to minutes, and fiat replenishment proceeds asynchronously over traditional rails; reverse flows work similarly. A direct mode bypasses the usual few-minute planning cycle for urgent wallet funding, trading netting efficiency for speed, while standard flows continue batching. The live infrastructure handles real traffic, with ongoing work focused on forecast-driven liquidity, faster replenishment through RTP and SEPA Instant, and broader product and corridor coverage.
ClickHouse’s ten-year open-source history is presented through a stained-glass analogy: many contributors remain visible as distinct pieces, while shared infrastructure holds the project together. The article connects that community account to production use through cBioPortal at Memorial Sloan Kettering, where moving heavy filtering into ClickHouse SQL made relevant paths about ten times faster, changing waits from minutes to seconds and supporting continued exploration. It also describes a demonstration built from bare-clone git history, one month of GH Archive events, and the shipped system.contributors table, all queried in ClickHouse. The account reports 100,000 issues and pull requests crossed on March 19, 2026, with 83% of issues and 99% of pull requests resolved at the time of the talk, and presents contributor credit and repair as operating parts of the project.
Speculative decoding is presented as the most important optimization for LLM inference, delivering speedups of 2x-3x rather than small percentage gains from kernel tuning. The technique accelerates the decode phase by using a lightweight draft model to generate candidate tokens that the target model processes in parallel, with lossless resampling. The authors released DFlash speculators for multiple Qwen models, achieving an additional 5-20% speedup over existing baselines, and worked with SGLang to integrate them. On a B200 node, the Qwen 3.5 122B-A10B model with speculation reaches over 1000 tok/s at concurrency 1, compared to 250 tok/s without speculation, demonstrating a 4x speedup. Training these speculators is simplified because the data-generating process is the target model itself, making data collection trivial, and the post envisions a flywheel where custom speculators and distillation enable continuous improvement in latency, cost, and quality for self-hosted inference.
METRO Markets outgrew its self-hosted Impala and Kudu-based Hadoop stack as the marketplace expanded, with parallel stores, inconsistent reporting, and too much maintenance burdening a lean data team. They evaluated BigQuery, Snowflake, and ClickHouse against production queries, finding faster ClickHouse performance, including one response falling from four seconds to under 200 milliseconds, while its pricing supported more predictable costs. Airbyte moves CDC data from MySQL and Postgres into ClickHouse; ReplacingMergeTree handles deduplication during background merges, and BigQuery-hosted Google Analytics data enters through cloud storage and external tables. The resulting warehouse consolidates company data and supports seller analytics, credit risk features through Query API endpoints, observability, and emerging AI use cases, while teams query a shared dataset through compatible tools; planned Kafka and Flink integration would extend it to streaming workloads.
Real-time analytics is presented as a foundational workload for customer-facing analytics, observability, fraud detection, operational applications, and AI agents rather than a niche use case. The discussion argues that performance claims require open, transparent, reproducible benchmarks that disclose methodology, datasets, queries, results, and operating conditions, including continuous ingestion, concurrency, memory pressure, and fresh data. A real-time database should support queryable data within milliseconds to a couple of seconds, incremental transformations, bounded tail latency as writes increase, configurable read-time versus write-time work, unified support for text, vector, JSON, and structured analytics, and efficiency as workload scales. The article connects renewed attention to agentic workloads, whose continuous, high-volume querying makes low latency, concurrency, and fresh context increasingly necessary, and calls for shared standards beyond a single latency number.
Appcues replaced an analytics stack that included Snowflake and Airflow after rising scale pushed queries to 20 seconds or more and limited reports to 10-minute windows. The company evaluated ClickHouse Cloud on AWS for real-time segmentation, exporting Snowflake data to Parquet in S3 and testing complex email segments with nested conditions, historical events, and negative logic. Production events now pass through Kafka and a clickwriter consumer, while a clickclient library built on Elixir’s Ecto generates controlled ClickHouse queries; SharedReplacingMergeTree, table engines, and materialized views were tuned during a one-year migration. Across 1.31 PB and 410 billion events, P95 queries fell below two seconds, ingestion latency dropped to about five seconds, and analytics costs decreased 23%, while new funnel reports and AI-powered insights were added.
Postgres managed by ClickHouse, which entered public beta in May, received a broad update covering console operations, per-service RBAC, Terraform, Stripe provisioning, ClickPipes migrations, billing, capacity reservations, and extensions. The redesigned console adds service health, resource, and query views, while RBAC provides View and Manage roles across the console and public REST API; the alpha Terraform provider covers lifecycle operations including resizing, HA, replicas, restore, and password rotation. Postgres-to-Postgres ClickPipes now automate schema migration, usage is integrated into ClickHouse Cloud’s metering and billing, and AWS capacity reservations are allocated dynamically across availability zones. Underneath, pg_clickhouse and pg_stat_ch moved to the header-only clickhouse-c library, removing a C++ exception conflict, using Postgres memory contexts, and reducing the x86-64 Linux extension binary from 4.9 MB to 1.4 MB; the update shipped less than a month after beta, with Terraform and extensions still evolving.
The best engineering reads, once a week.
