---
title: "Autonomous Recovery"
description: "1 posts about Autonomous Recovery, summarised, each linking to the original."
---

# Autonomous Recovery
> 1 posts about Autonomous Recovery, summarised, each linking to the original.

## Articles

### [How NVIDIA Mission Control 2.5 Brings Autonomous Recovery to Kubernetes](https://yomu.fyi/post/how-nvidia-mission-control-2-5-brings-autonomous-recovery-to-kubernete.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Oct 26, 2026
- Type: Announcement

NVIDIA Mission Control 2.5 extends its autonomous recovery engine, previously limited to Slurm-based HPC, to Kubernetes workloads managed via NVIDIA Run:ai for AI factories, coordinating three separate services: the scheduler for workload placement, the job recovery service for failure-context decisions, and the hardware recovery service for node isolation and diagnostics. When a hardware fault occurs, job recovery restarts the workload on healthy nodes while hardware recovery concurrently isolates the suspect GPU. For transient software errors, only job restart occurs without hardware quarantine. Early degradation signals cause proactive node removal before job scheduling. This consistent recovery model across Kubernetes and Slurm reduces MTTR and improves GPU utilization by shifting from reactive manual intervention to proactive self-healing, benefiting both platform teams and AI practitioners.
