Faster Pull Request Merges

Summary

Ramp’s growing Python pytest suite took over 12 minutes, while pull requests could wait up to one to two hours between merge queueing and production. Previously, Kodiak kept pull requests in a merge queue, requiring each PR to be up-to-date with master and rerun the full suite after every merge. Removing that requirement exposed a failure mode: parallel branches could introduce Alembic migrations with multiple heads, making the database migration history invalid. Ramp instead added migrations/migration-hash.txt, a hash of migration filenames that creates merge conflicts between migration-changing PRs while leaving other PRs free to merge; the deploy pipeline still runs tests on master before deployment. This reduced average merge time from more than an hour to 12 minutes.

Context

Ramp’s Python test suite had grown to more than 12 minutes per run, and pull requests waited up to one to two hours in the merge queue. The team wanted to eliminate the queue without allowing independently passing changes to break deployments, particularly through conflicting Alembic database migrations.

Approach / What changed

Pull requests without database migrations were allowed to merge without being up-to-date with master, while migration pull requests were prevented from merging through a generated migrations/migration-hash.txt file. The file hashes migration filenames, so concurrent migration changes produce a merge conflict. The deploy pipeline also requires tests to pass on master before deployment.

Takeaways

• Alembic cannot determine a single database head when two migrations branch from the same revision and are merged independently.
• Hashing the filenames in migrations/versions makes concurrent migration changes conflict without blocking pull requests that do not modify migrations.
• Average merge time fell from more than one hour to 12 minutes, matching the stated test-suite runtime.

Related reading