Machine-learning predictive autoscaling for Flink

Summary

Grab’s Flink platform saw application count grow 2.5 times, making cost-effective, self-service CPU provisioning difficult for users who often relied on trial and error and over-provisioned resources. Reactive autoscaling with Adaptive Scheduler, Kubernetes HPA, and metrics such as CPU usage or consumer latency caused restart spikes, scaling spirals, fluctuations, and additional constraints when connectors such as Kafka reached fixed partition parallelism. Predictive Resource Advisor instead uses vertical scaling and predictive, deterministic CPU decisions, automatically collecting data, training models, and applying configuration changes. The source says applications using it experienced fewer autoscaling-related disruptions, adapted to workload growth and surges, and averaged more than 35% cloud infrastructure cost savings. It also reports simpler default-configuration deployment, while planned work includes memory tuning, improved burst handling, and more efficient model training.

Takeaways

1. 1

   Flink restarts reprocess records from the last checkpoint and handle newly backlogged records, causing temporary CPU and consumer-latency spikes after scaling events.

2. 2

   Kafka’s fixed topic partitions limit horizontal consumer parallelism; once that limit is reached, increasing CPU or memory per instance becomes necessary to raise throughput.

3. 3

   Applications using Predictive Resource Advisor averaged more than 35% savings in cloud infrastructure cost, with CPU provisioning aligned more closely to requirements during low-traffic periods.