AI is approving our pull requests: Here’s how we made it safe

Summary

Intercom describes how it uses an AI PR review Agent to approve some pull requests amid a workflow where more than 93% are Agent-driven and engineers deploy hundreds of times daily. The system decomposes review into specialist sub-Agents that assess problem descriptions, intent alignment, safety, logic, best practices, and execution paths, while applying Intercom-specific guidance and enforcing smaller changes. In a pilot of more than 100 PRs, AI-approved changes had zero reverts and approval time improved 6–16 times at the 75th percentile; in four rollout weeks, 497 PRs went fully autonomous. Reported revert rates were lower for AI-authored than human-authored code in both backend and frontend comparisons, though the source says zero reverts will not necessarily persist. The process remains auditable, allows human review requests, and keeps the shipping engineer accountable for production behavior and rollback.

Takeaways

1. 1

   The Agent reviews separate dimensions of a pull request, including problem quality, intent alignment, safety, logical correctness, best practices, and execution paths.

2. 2

   A pilot covering more than 100 PRs reported zero reverts for AI-approved changes and a 6–16x improvement in time-to-approval at the 75th percentile.

3. 3

   Every AI-approved PR is labelled, logged, and queryable, with review comments, approval decisions, test results, and merge events recorded for auditability.