Enabling Evolutionary Database Development: database branching with Lakebase, continued

Summary

This installment revisits Evolutionary Database Design and argues that Databricks Lakebase removes the infrastructure constraints that kept several of its database-change practices aspirational. Lakebase is a managed Postgres database with compute separated from shared durable storage, while copy-on-write branches create a new pointer and divergence marker in roughly one second without copying parent data. That enables per-developer, per-PR, and per-experiment databases, real Postgres test branches instead of mocks or in-memory substitutes, and CI isolation at pull-request granularity. The updated playbook adds idempotent migrations, destructive testing, and database-level A/B prototyping, with Unity Catalog governance inherited by branches and agents receiving branches rather than production access. Jen’s example corrupts production-shaped data to test an inventory-code migration and compares column and lookup-table designs, rejecting the latter because its common read path requires a join.

Takeaways

1. 1

   Lakebase branches are metadata operations that share unchanged storage with their parent and diverge only on modified pages, completing in roughly one second regardless of parent size.

2. 2

   Per-branch Postgres environments replace mocks, H2, SQLite, and shared staging for database-change testing, while enabling destructive tests with a zero-cost reset.

3. 3

   Parallel database branches let teams compare schema designs against production-shaped data; Jen rejected a lookup table because its common inventory read path required a join.