Loading…
Branching databases like code: a CI/CD pattern for Lakebase, in production at Glaspoort
Hadi Farhat, Gideon Spierings, Ricardo de Vries, Raymon Veldman
- Source
- Databricks
- Published
- Added to Yomu
Summary
Glaspoort replaced a sprawl of outdated, one-off BI reports with a custom application that combines Databricks analytical tools with Lakebase, a serverless Postgres OLTP database. The application keeps curated lakehouse data in a read-only schema while application state occupies a separate schema, and supports development, acceptance, and production environments. Its CI/CD pattern treats database branches like code: long-lived environments branch directly from production, each pull request receives an ephemeral production-shaped branch, migrations are replayed, and a staging app image runs the full test suite against it. This topology makes periodic resets cheap, avoids deleting dependent branches, and preserves fresh test data; the team chose stacked promotion for velocity, retaining a separate crisis pipeline for urgent fixes.
Context
Glaspoort’s reporting process produced one-off reports whose questions were outdated when completed, leaving users without a way to pursue follow-up questions. The database workflow also risked stale environments, shared-database conflicts, and costly resets caused by branches depending on one another.
Approach / What changed
The design uses Lakebase branches as database counterparts to Git branches. Development and acceptance branch directly from production, while each pull request gets a disposable production-shaped branch where migrations and the application image are tested together before gated promotion. Database migrations serve as the single source of truth for schema changes.
Takeaways
- Development and acceptance are sibling branches of production rather than parent and child; resetting either branch therefore does not require deleting dependent environments, rewiring connection strings, or reapplying inherited grants.
- CI creates a fresh production-based branch for each PR, replays migrations when the Git diff requires it, deploys the new application image to a staging slot, and runs the full test suite before the branch is archived when the PR closes.
- The team chose stacked promotion because it suits overlapping work in a small team; every promotion gate validates stacked migrations against a fresh production fork, while a separate crisis pipeline provides an urgent-fix path.