---
title: "Autoscaling Is a Control Loop, Not a Button"
description: "Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload."
---

# Autoscaling Is a Control Loop, Not a Button

**Type:** [Trade-offs](https://yomu.fyi/visual-explainers/category/trade-offs.md)

![A trade-off diagram compares reactive and predictive autoscaling, horizontal and vertical capacity changes, and the conditions that favor each approach.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/0d1f3331-f81f-42e4-9f3f-5fee8f502304/image-fe7efc37-551d-4a28-8aaf-087528e8559a.webp)

> Autoscaling trades responsiveness, stability, and cost: this explainer shows how signal choice, scaling direction, and capacity limits shape the right control loop for a workload.

## Explanation

Autoscaling adjusts capacity as demand changes, but the decision is a control-loop trade-off rather than a single feature. Reactive signals such as CPU, queue, concurrency, or latency can respond quickly yet oscillate or arrive after pressure builds; predictive approaches can reduce disruption but depend on workload history and tuning. The explainer connects scaling direction, safety checks, warm capacity, and node or partition limits to the conditions that make each strategy useful.

## FAQ

### Why can a reactive autoscaler become unstable?

Reactive scaling follows signals such as CPU usage, queue depth, concurrency, or latency after demand changes. The supplied Flink example reports restart spikes, scaling spirals, and fluctuations when reactive signals were poorly matched to workload behavior.

### When should scaling be horizontal versus vertical?

Horizontal scaling adds replicas or nodes and is useful when work can be spread across capacity, as in MachineSet, model-serving, and Spark examples. Vertical scaling changes the resources available to an existing workload and can fit stateful or resource-sensitive jobs, but it requires careful capacity planning and safe resizing behavior.

### What safety checks matter when scaling down?

Downscaling should consider state, working-set memory, active connections, and the time needed for capacity changes. Lakebase uses checks to prevent unsafe downscales, while its decoupled compute and durable storage make live resizing possible without moving the database.

**Tags:** [Autoscaling](https://yomu.fyi/topic/autoscaling.md), [Cost Optimization](https://yomu.fyi/topic/cost-optimization.md), [Kubernetes](https://yomu.fyi/topic/kubernetes.md), [Monitoring](https://yomu.fyi/topic/monitoring.md), [Scalability](https://yomu.fyi/topic/scalability.md)

## Related reading

### [Red Hat OpenShift autoscaling using MachineSet autoscaler with KEDA](https://yomu.fyi/post/red-hat-openshift-autoscaling-using-machineset-autoscaler-with-keda.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Ramon Gordillo Gutierrez, Jose Ortiz Padilla
- Published: Sep 1, 2026
- Type: Tutorial

The demonstration compares OpenShift MachineSet Autoscaler with KEDA against the previously used Cluster Autoscaler, using metrics to scale a MachineSet before pods remain pending. After installing Red Hat’s Custom Metrics Autoscaler operator (KEDA 2.19), granting permissions for MachineSet scaling and Thanos/Prometheus access, it creates a ScaledObject with a Prometheus trigger targeting demo-p4p95-worker-eastus3. Because a MachineSet has no pods, the HPA path cannot use metricType: Value, so AverageValue is paired with a PromQL query that multiplies CPU request utilization by the current node count. With nine one-CPU test pods, the MachineSet scaled from one to two to three nodes as the metric crossed 75%, and all pods reached Running; two additional Standard\_D4s\_v3 VMs were provisioned in zone 3.


### [Autoscaling Lakebase Postgres](https://yomu.fyi/post/autoscaling-lakebase-postgres.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Carlota Soto
- Published: Aug 31, 2026
- Type: Explainer

Lakebase Postgres addresses the waste and uncertainty of choosing a fixed database instance size before workload demand is known. Its decoupled architecture separates stateless compute from durable storage, allowing compute to start, stop, move, or resize without moving the database. The autoscaling algorithm combines CPU load, memory pressure, and a time-aware estimate of the Postgres working set, using modified HyperLogLog registers that store timestamps and estimate windows from one to 60 minutes. Scaling occurs live through autoscaler-agent, vm-monitor, Kubernetes scheduling, and NeonVM, with checks preventing unsafe downscales and scheduler approval for upscales. The system scales in both directions without dropping connections and operates across 100-millisecond, five-second, and 20-second timescales, including more than 32,000 resizes per month.


### [AI Serving Platform That Adapts to Your Model](https://yomu.fyi/post/ai-serving-platform-that-adapts-to-your-model.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Anshul Gupta
- Published: Jun 10, 2026
- Type: Problem & solution

Databricks Custom Model Serving addresses the operational burden of serving custom models, whose resource profiles, traffic patterns, and latency requirements vary widely from small CPU classifiers to large GPU-backed language models. Its fully managed platform packages MLflow models and uses isolated Kubernetes deployments, model-appropriate runtimes, and a short request path to limit interference and per-request overhead. At the center, the AutoPilot Pod Autoscaler combines active concurrency and queue signals for horizontal scaling with CPU, GPU, and memory measurements for model-aware target-concurrency adjustment, allowing one controller to adapt across workloads. Warm pools, provisioned concurrency, and zero-downtime updates address startup and deployment concerns, while reported production results include 90%+ cost savings for some customers, up to 2x improvement in p99 and p50 latency, 100K+ QPS, and 99.99% availability.


### [Rethinking Distributed Systems for Serverless Performance and Reliability](https://yomu.fyi/post/rethinking-distributed-systems-for-serverless-performance-and-reliabil.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Aaron Davidson, Roland Fäustlin, Zach Williams
- Published: May 6, 2026
- Type: Explainer

Building serverless compute for Apache Spark requires more than warm machine pools or basic autoscaling because traditional deployments couple user applications to infrastructure, allowing contention and inefficiencies to undermine performance and reliability. The proposed architecture separates these concerns through Spark Connect’s client-server model over gRPC, a gateway that routes workloads using query size, cluster utilization, and latency profile, and an adaptive autoscaler that adjusts capacity horizontally and vertically. Spark Connect isolates user applications from drivers, while the gateway continually re-evaluates placement to reduce interference between workloads. The autoscaler offers Standard and Performance-Optimized modes and can respond to out-of-memory errors by restarting tasks on larger VMs without manual intervention. Reported outcomes include a 99.998% upgrade success rate across more than 4.5 billion workloads, 2–5x faster Unilever pipelines, and operational-cost reductions of 25%.


### [Machine-learning predictive autoscaling for Flink](https://yomu.fyi/post/machine-learning-predictive-autoscaling-for-flink.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Minh Nhat Nguyen
- Published: Oct 30, 2025
- Type: Problem & solution

Grab’s Flink platform saw application count grow 2.5 times, making cost-effective, self-service CPU provisioning difficult for users who often relied on trial and error and over-provisioned resources. Reactive autoscaling with Adaptive Scheduler, Kubernetes HPA, and metrics such as CPU usage or consumer latency caused restart spikes, scaling spirals, fluctuations, and additional constraints when connectors such as Kafka reached fixed partition parallelism. Predictive Resource Advisor instead uses vertical scaling and predictive, deterministic CPU decisions, automatically collecting data, training models, and applying configuration changes. The source says applications using it experienced fewer autoscaling-related disruptions, adapted to workload growth and surges, and averaged more than 35% cloud infrastructure cost savings. It also reports simpler default-configuration deployment, while planned work includes memory tuning, improved burst handling, and more efficient model training.