Announcing On-Demand Compute: Instant compute for your most intensive workloads

Summary

ClickHouse Cloud’s On-Demand Compute is announced in private preview as a way to run eligible queries on a shared pool of workers outside a service’s primary cluster, avoiding production contention and reactive autoscaling delays. It uses ClickHouse’s multistage distributed query execution and cost-based optimizer to distribute complex workloads, with queries requesting worker counts through settings including make_distributed_plan and distributed_plan_workers_num. Workers are leased for at least 60 seconds, reused during the lease, and released when it expires; concurrent queries share the pool, which grows to the largest request. In a stated benchmark, On-Demand Compute was faster for the tested workload and at 25 concurrent queries delivered similar performance while using 50% less compute, though results vary by query shape.

Takeaways

1. 1

   On-Demand Compute supports SharedMergeTree, Apache Iceberg, and Delta Lake workloads, including ad hoc queries, selected read offloading, and eligible data lake queries.

2. 2

   Workers have a minimum 60-second lease that renews for longer-running queries; subsequent queries can reuse active workers without a cold start or discovery delay.

3. 3

   In the benchmark, 25 concurrent queries using 15 workers achieved performance similar to the stateful cluster’s single-node execution while using 50% less compute; results vary by workload.