Same Cluster, 33 Points More Utilization: What Changed Was the Order

Summary

The post presents a constraint-aware GPU allocator as a response to utilization losses caused by fixed real-time reservations and FIFO ordering under contention. It compares the allocator with FIFO on identical hardware and workloads across seven benchmark scenarios, modeling batch-like jobs as contiguous GPU blocks and real-time inference as a timestep-by-timestep demand curve with bounded GPU swaps. Across five contention scenarios, utilization improved from 52–85% to 72–88%, while priority-weighted value increased 24.6–105.1%; the strongest case gained 33 percentage points of utilization and 105% value, and the scale test gained 15.9% value at identical utilization and throughput. The scheduler optimizes a 24-hour horizon but commits only the current timestep, re-running every 30–60 minutes so fresh observations absorb forecast error while preserving running work.

Takeaways

1. 1

   Fixed real-time reservations can leave GPUs idle but unavailable to batch work; in reservation-dominated scenarios, FIFO utilization was 51.6% and 53.6%.

2. 2

   The scale test produced identical 44.9% utilization and 27 of 30 completed jobs under both schedulers, yet the allocator delivered 15.9% more priority-weighted value.

3. 3

   Re-optimizing a 24-hour plan every 30–60 minutes while committing only the current timestep is intended to absorb forecast error and avoid end-of-horizon decisions that damage near-term allocations.