Loading post…
ClickhouseTutorial
Monitor SLAs and scale ClickHouse Cloud with clickhousectl and agents
Summary
ClickHouse Cloud can automatically scale infrastructure up and down, but teams may need to monitor latency SLAs for specific workloads such as a frontend dashboard requiring p99 under 200 ms. The guide tags queries with log_comment, reads system.query_log through clickhousectl, and aggregates query_duration_ms over a defined window to measure the SLA. When it is breached, operators compare query volume and latency with Prometheus-compatible CPU, memory, and concurrency metrics to distinguish increased concurrency from resource contention, then choose horizontal replica scaling or vertical replica sizing. A cron-based script can scale out when p99 exceeds a threshold, while an agent can inspect evidence and apply one scaling action using clickhousectl; actions are recorded in the ClickHouse Cloud activity log.
Context
Specific query SLAs, such as a frontend dashboard requiring p99 latency under 200 ms, may need monitoring beyond ClickHouse Cloud's automatic infrastructure scaling. An SLA breach shows that latency increased, but additional investigation is needed to determine whether query concurrency, competing workloads, CPU, memory, or other resource conditions are responsible.
Approach / What changed
Tag tracked queries with log_comment and measure their latency from system.query_log using clickhousectl. Investigate breaches by comparing query volume and p99 latency with service metrics from the Prometheus-compatible endpoint, then scale horizontally with more replicas or vertically by changing replica memory according to the suspected cause. The workflow can run from cron or delegate investigation and remediation to an agent, with actions recorded in the activity log.
Takeaways
Use log_comment to isolate a workload in system.query_log and calculate its p99 query_duration_ms over a chosen time window.
High concurrency with low memory points toward adding replicas, while memory near the limit on every replica suggests using larger replicas.
The workflow supports both a simple cron-based threshold check and agent-driven investigation; clickhousectl actions appear in the ClickHouse Cloud activity log.
Source and freshness
- Source
- clickhouse.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.