Backstage with Lakebase

Summary

Backstage with Lakebase describes a proof of concept that moves Backstage’s stateful internal developer portal from standard Postgres to Databricks Lakebase, a serverless Postgres surface in the Databricks Data + AI Platform. Connection required an app-config.yaml change, PgSearchEngine instead of the default in-memory search, and short-lived OAuth JWT credentials generated with databricks postgres generate-database-credential; Knex migrations then ran cleanly. Using copy-on-write branches, the team cloned a roughly 63 MB catalog in 1.09 seconds and recovered 32 deleted entities in 3.78 seconds, while observing that point-in-time recovery snaps backward to the nearest WAL record. The post argues that cheap, isolated database state can replace much mocking and staging iteration with real-data development, branch-based testing, schema validation in CI, and automated cleanup, while noting that VS Code/Cursor synchronization tooling remains underway.

Context

The proof of concept tests whether the traditional separation between operational and analytical databases is still necessary when storage is shared, compute is isolated per workload, and governance is handled at the catalog layer. Backstage was selected partly because its schema migrations are described as fragile, making it a useful test of database branching and recovery.

Approach / What changed

Backstage was connected to Lakebase through its Postgres wire protocol by updating app-config.yaml, using PgSearchEngine, and supplying scoped OAuth JWT credentials. The proof of concept tested copy-on-write branching, point-in-time recovery after deleting final_entities, and a workflow in which database branches accompany Git branches, development, CI, QA, and deployment.

Takeaways

• Lakebase rejected classic Databricks Personal Access Tokens; the proof of concept used databricks postgres generate-database-credential to create scoped, short-lived OAuth JWTs and refreshed the token every 50 minutes with a cron script.
• A copy-on-write branch cloned the approximately 63 MB Backstage catalog in 1.09 seconds. Point-in-time recovery restored 32 deleted entities in 3.78 seconds, but recovery snapped backward to the nearest WAL record rather than the exact requested timestamp.
• The proposed workflow connects development and CI to real database branches, reducing reliance on mocks. The post reports that mock objects account for 20–30% of test code across the partner teams it has experienced with.

Related reading