Loading…
Topic
4 posts about Scheduling. Every summary links to the original.
See the idea
A five-checkpoint workflow for assignment problems: write the spec, name the objective, test whether choices interact, pick a solver class by scale, then validate against live state.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Ai2 replaced its priority-based GPU cluster scheduler with a system combining GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract to prioritize high-impact research while maintaining full occupancy. The previous priority scheduler induced GPU squatting, priority inflation, and constant negotiation over non-preemptable workloads, exemplifying a tragedy of the commons. In the new model, managers allocate proportional GPU time budgets to projects, making resource allocation a transparent administrative process rather than an operational debate. The scheduler tracks occupancy over a time window with weights set by budgets, and time-slicing enforces minimum runtime protection before workloads become preemptible. However, interactive sessions suffered from the 8-hour cap, prompting investment in a CPU-only cluster and restorable sessions, while potential capacity fragmentation for large workloads remains under investigation.
Investing in real-time customer support can drive revenue and satisfaction, but scaling it requires careful operational planning. The team found that new customers were 30% more likely to start a trial after a swift first response, and they defined two key metrics: first response time (target under 2 minutes) and subsequent response time. They started by offering real-time support to 50% of new customers, recruiting tenured top performers as Project Leads to help roll out the experiment and identify effective workflows. Operational success depended on four elements: scheduling (using heatmaps of conversation volume and dedicated tools), tools (Intercom’s Messenger and Inbox), workflows (shifting from pass-back to 100% ownership), and communication (Slack channels for handovers and feedback). After refining qualification criteria based on rep feedback to exclude customers with previously activated apps, the team scaled real-time support to 100% of new customers within six months.
AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.
ThunderAgent is a program-level scheduler for high-throughput agentic inference, developed by researchers at Georgia Tech, UIUC, CMU, and Together AI. It solves KV cache thrashing, which occurs when request-level engines like vLLM and SGLang evict an agent’s KV cache during tool calls and recompute it on resume, creating a cascade of evictions. ThunderAgent tracks each workflow as a schedulable program, pausing low-priority workflows under memory pressure and routing resumed programs through a global waiting queue to the node with the most available capacity. In synthetic data generation pipelines, it achieves up to 2.5x single-node throughput and 2.4x speedup on 8 nodes with near-linear scaling, and was accepted as an ICML 2026 Spotlight paper.
The best engineering reads, once a week.
