---
title: "Fixed cadence to seconds: making ClickHouse Cloud autoscaling more reactive"
description: "ClickHouse Cloud’s vertical autoscaling recommender originally ran on a fixed timer, creating latency when workloads suddenly increased and services needed resources before the next scheduled pass. The redesign keeps the periodic sweep as a safety net while adding a reactive fast path, using controller-runtime as an event-processing layer with sources feeding a shared rate-limited work queue and reconciler. The queue provides deduplication, exponential backoff, bounded concurrency, leader election, and metrics, while ClickHouse stores near-real-time signals that a polling source reads; the excerpt notes polling sets a floor on reaction speed. The reactive path is rolling out cautiously in observe-only mode, with periodic reconciliation remaining the main driver, and the stated goal is to trigger scale-ups for critical events such as out-of-memory conditions in seconds rather than waiting for the next scheduled run."
---

# Fixed cadence to seconds: making ClickHouse Cloud autoscaling more reactive

[Clickhouse](https://yomu.fyi/company/clickhouse) · Aug 4, 2026

**Type:** Problem & solution

## Summary

ClickHouse Cloud’s vertical autoscaling recommender originally ran on a fixed timer, creating latency when workloads suddenly increased and services needed resources before the next scheduled pass. The redesign keeps the periodic sweep as a safety net while adding a reactive fast path, using controller-runtime as an event-processing layer with sources feeding a shared rate-limited work queue and reconciler. The queue provides deduplication, exponential backoff, bounded concurrency, leader election, and metrics, while ClickHouse stores near-real-time signals that a polling source reads; the excerpt notes polling sets a floor on reaction speed. The reactive path is rolling out cautiously in observe-only mode, with periodic reconciliation remaining the main driver, and the stated goal is to trigger scale-ups for critical events such as out-of-memory conditions in seconds rather than waiting for the next scheduled run.

## Context

The original recommendation service processed every service on a fixed schedule. This made scale-up decisions wait until the next scheduled pass, even after urgent events such as an out-of-memory condition. The periodic sweep also remains important for continuously verifying usage, adjusting resources, and scaling down when reasonable to avoid paying for unused capacity.

## Approach / What changed

The orchestration layer was rebuilt around controller-runtime. A periodic source and a reactive source feed one shared rate-limited work queue, which deduplicates service keys, applies exponential backoff, and bounds concurrency before a shared idempotent reconciler determines the appropriate service size. Near-real-time signals are read from a ClickHouse table through polling, while periodic reconciliation remains the safety net. The reactive path is being introduced in observe-only mode before it drives decisions.

## Takeaways

- The shared controller-runtime queue separates trigger logic from job-processing concerns, so periodic and reactive sources do not need to coordinate deduplication, retries, or concurrency themselves.
- The signal source polls a table rather than consuming a push stream, making its polling interval the lower bound on reaction speed; a restart does not permanently lose an event because later queries can enqueue the service again.
- The design favors idempotent, inexpensive recomputation and periodic self-correction because signal volume is modest; guaranteed delivery, strict ordering, or stateful-window correlation would likely require a dedicated streaming platform.

**Tags:** [Kubernetes](https://yomu.fyi/topic/kubernetes), [Performance](https://yomu.fyi/topic/performance), [Reliability](https://yomu.fyi/topic/reliability), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [Clickhouse](https://clickhouse.com/blog/making-clickhouse-cloud-autoscaling-more-reactive)
- Source URL: https://clickhouse.com/blog/making-clickhouse-cloud-autoscaling-more-reactive
- Ingested by Yomu: 2026-08-29T04:00:54.178Z

[Read original post](https://clickhouse.com/blog/making-clickhouse-cloud-autoscaling-more-reactive)