Better tools made Copilot code review worse. Here’s how we actually improved it.

Summary

When migrating GitHub Copilot code review from custom navigation tools to shared Unix-style CLI utilities—grep, glob, and view—benchmarks revealed higher review costs and fewer caught issues. Rather than an issue with the underlying tools, trace analysis showed the agent used general-purpose coding assistant instructions that triggered expansive repository browsing loops. In response, the team rewrote the tool guidance to enforce a review-specific workflow anchored to pull request diffs. The updated instructions direct the agent to narrow candidate call sites and files using batched discovery before reading minimal line ranges with view. This workflow tuning reduced average review costs by approximately 20% in production while maintaining review quality.

Takeaways

1. 1

   Retaining broad tool output inside an LLM's working context inflates token usage and degrades focus across subsequent reasoning steps.

2. 2

   Restructuring tool guidance to prioritize diff-anchored discovery before focused reads lowered average review costs by roughly 20% while maintaining review quality.

3. 3

   Tool prompt optimizations tailored to pull request reviews failed to produce identical wins in the Copilot CLI, where broad interactive exploration remains necessary.