Topic

Slurm

2 posts about Slurm. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

NVIDIA
Topology-Aware Workload Scheduling with NVIDIA Topograph
AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.

NVIDIA
Scaling Federated Learning Across Docker, Kubernetes, and Slurm with NVIDIA FLARE
NVIDIA FLARE addresses the operational challenge of running federated learning across sites that use different infrastructure, including Docker, Kubernetes, and Slurm. Its two-layer architecture keeps long-running server and client parent processes available for authentication and coordination while launching transient job workers through each site’s native runtime. Jobs declare portable GPU, CPU, and host-memory requirements, which local launchers translate into Docker containers, Kubernetes pods, or Slurm allocations; studies add scoped membership and site-owned mappings for datasets, images, secrets, and scheduling policy. The result is a heterogeneous federation where organizations retain local resource authority, can allocate compute on demand, and run multiple studies without requiring one shared platform.

The best engineering reads, once a week.

Topic

2 posts about Slurm. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

NVIDIA
Topology-Aware Workload Scheduling with NVIDIA Topograph
AI factories are power-limited systems where GPU workload placement is a key optimization; poor placement fragments topology domains and forces traffic across shared links, reducing throughput and raising job costs. NVIDIA Topograph discovers cluster topology from cloud APIs or on-premises fabric systems, normalizes it into a common model, and publishes it in formats each workload manager expects, such as Kubernetes node labels, Slurm topology configuration, or Slinky ConfigMaps. It uses five components: API Server, Node Observer, Node Data Broker, Provider, and Engine to keep the topology view current as the cluster changes. The post covers deploying Topograph with Helm on Kubernetes, as a system package on Slurm, and as a Helm chart on Slinky, with support for cloud providers including Google Cloud, Lambda, Nebius, Nscale, and OCI. Topograph works alongside Dynamic Resource Allocation and KAI Scheduler to enable topology-aware gang scheduling, improving AI factory efficiency and tokens per watt.

NVIDIA
Scaling Federated Learning Across Docker, Kubernetes, and Slurm with NVIDIA FLARE
NVIDIA FLARE addresses the operational challenge of running federated learning across sites that use different infrastructure, including Docker, Kubernetes, and Slurm. Its two-layer architecture keeps long-running server and client parent processes available for authentication and coordination while launching transient job workers through each site’s native runtime. Jobs declare portable GPU, CPU, and host-memory requirements, which local launchers translate into Docker containers, Kubernetes pods, or Slurm allocations; studies add scoped membership and site-owned mappings for datasets, images, secrets, and scheduling policy. The result is a heterogeneous federation where organizations retain local resource authority, can allocate compute on demand, and run multiple studies without requiring one shared platform.

The best engineering reads, once a week.