Carving The Scheduler Out Of Our Orchestrator

Summary

Fly.io describes why it is carving scheduling out of its existing Nomad-based orchestrator and replacing it with a simpler system centered on flyd and AppsV2. It first reduces orchestration to a client-server supervisor, then explains centralized scheduling: workers register resources, while a server filters candidates by constraints and ranks them. The proposed design avoids jobs sitting pending or being placed in nearby but incorrect regions; requesting VMs in MAD yields MAD capacity or nothing. Policy-heavy behavior such as deployment spares, crash recovery across workers, and regional decisions moves into flyctl and other platform components rather than the scheduler. Fly Machines already exposes APIs to reserve, start, and stop individual VMs, with Nomad still used in January 2023 and flyd expected for deployments in December 2023.

Takeaways

1. 1

   Orchestration is presented as a client-server process supervisor: workers poll for jobs, claim them, run them, and release them through HTTP endpoints.

2. 2

   The centralized scheduler registers worker capacity, filters candidates by constraints such as CPU and disk space, and ranks the survivors instead of having workers compete to claim jobs.

3. 3

   AppsV2 moves complex deployment and recovery decisions into flyctl and other platform components, while flyd exposes primitives to reserve, start, and stop individual VMs.