How Databricks Uses AI to Accelerate Incident Investigation

Summary

Databricks describes AI SRE, an AI-powered debugging agent for operating hundreds of microservices across more than 1,500 Kubernetes clusters, 70+ regions, and three clouds. It addresses incident investigations by assembling context from platform health, service telemetry, changes, dependencies, and team-specific runbooks, with automatic triage followed by interactive, natural-language investigation. The platform uses layered primitives, purpose-built APIs, and a core orchestration engine; team-owned agentic runbooks encode domain procedures, while controlled access and evidence links support auditing. The post reports support for more than 150 teams, 250+ weekly active users, and over 2,000 daily investigations, with users saving several hours of debugging time.

Takeaways

1. 1

   AI SRE’s automatic triage runs platform health checks, service analysis, and runbook execution in parallel before the on-call engineer starts investigating.

2. 2

   Teams can convert existing runbooks into agentic runbooks using skills that draw on code, observability data, and past incident history.

3. 3

   The system emphasizes traceable evidence and graceful degradation: when confidence is insufficient, it presents gathered evidence without asserting an unsupported root cause.