---
title: "Same Cluster, 33 Points More Utilization: What Changed Was the Order"
description: "The post presents a constraint-aware GPU allocator as a response to utilization losses caused by fixed real-time reservations and FIFO ordering under contention. It compares the allocator with FIFO on identical hardware and workloads across seven benchmark scenarios, modeling batch-like jobs as contiguous GPU blocks and real-time inference as a timestep-by-timestep demand curve with bounded GPU swaps. Across five contention scenarios, utilization improved from 52–85% to 72–88%, while priority-weighted value increased 24.6–105.1%; the strongest case gained 33 percentage points of utilization and 105% value, and the scale test gained 15.9% value at identical utilization and throughput. The scheduler optimizes a 24-hour horizon but commits only the current timestep, re-running every 30–60 minutes so fresh observations absorb forecast error while preserving running work."
---

# Same Cluster, 33 Points More Utilization: What Changed Was the Order

[Hugging Face](https://yomu.fyi/company/hugging-face) · Gabriel Pimenta de Freitas Cardoso, Breno de Almeida Beleza, Francisco de Almeida Rocha Alves, Bruno Duarte · Aug 17, 2026

**Type:** Benchmark

## Summary

The post presents a constraint-aware GPU allocator as a response to utilization losses caused by fixed real-time reservations and FIFO ordering under contention. It compares the allocator with FIFO on identical hardware and workloads across seven benchmark scenarios, modeling batch-like jobs as contiguous GPU blocks and real-time inference as a timestep-by-timestep demand curve with bounded GPU swaps. Across five contention scenarios, utilization improved from 52–85% to 72–88%, while priority-weighted value increased 24.6–105.1%; the strongest case gained 33 percentage points of utilization and 105% value, and the scale test gained 15.9% value at identical utilization and throughput. The scheduler optimizes a 24-hour horizon but commits only the current timestep, re-running every 30–60 minutes so fresh observations absorb forecast error while preserving running work.

## Context

The work addresses GPU capacity losses caused by fixed reservations for fluctuating real-time inference and FIFO placement of batch-like jobs. Under contention, FIFO can commit capacity without considering job priority, future fit, or the differing shapes and durations of training, batch inference, quantization, and real-time workloads.

## Approach / What changed

A constraint-aware allocator models allocation across GPUs and timesteps. It treats real-time demand as a forecast curve, places batch-like jobs in contiguous power-of-two GPU blocks, limits real-time GPU swaps between timesteps, preserves running work, and optimizes priority-weighted batch rewards against real-time shortfall penalties. It plans over 24 hours, commits only the current timestep, and re-runs every 30–60 minutes using fresh data.

## Takeaways

- Fixed real-time reservations can leave GPUs idle but unavailable to batch work; in reservation-dominated scenarios, FIFO utilization was 51.6% and 53.6%.
- The scale test produced identical 44.9% utilization and 27 of 30 completed jobs under both schedulers, yet the allocator delivered 15.9% more priority-weighted value.
- Re-optimizing a 24-hour plan every 30–60 minutes while committing only the current timestep is intended to absorb forecast error and avoid end-of-horizon decisions that damage near-term allocations.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

- Source: [Hugging Face](https://huggingface.co/blog/Dharma-AI/gpu-management-pt2)
- Source URL: https://huggingface.co/blog/Dharma-AI/gpu-management-pt2
- Ingested by Yomu: 2026-08-29T05:41:28.702Z

[Read original post](https://huggingface.co/blog/Dharma-AI/gpu-management-pt2)