ClickGap: Autonomous QA for ClickHouse

Summary

ClickGap addresses defects that survive ClickHouse’s extensive CI, human review, and in-house automated review, including untested behavior, silent resource waste, and performance regressions. The autonomous QA daemon examines eligible merged pull requests against verified builds, uses separate bug and coverage analysts, and requires hypotheses to become executed tests. An adversarial reviewer and ten evidence gates check consequences, source citations, complete-file review, reproducibility, and other conditions before a report can be filed. Five months after launch, ClickGap had filed roughly 500 issues and opened around 200 coverage or fix pull requests; more than half of the issues were closed, with over 200 linked to a fix PR. The system’s central design goal is to make reports trustworthy rather than merely prolific.

Takeaways

1. 1

   ClickGap reviews merged code because that is what ships, and its findings arrive within hours of a merge.

2. 2

   A skip index defect returned identical query results while causing a query to read all 16 granules instead of one, demonstrating why result assertions alone miss performance failures.

3. 3

   Coverage findings must identify a single-line mutation that the new test catches and show that the original pull request’s tests remain green after the mutation.