Backstage with Lakebase, part 3

Summary

This third part describes joining Backstage’s live operational Postgres catalog with analytical billing data so teams can query resource ownership and cloud spend together. Lakebase isolates compute per workload, allowing portal traffic and FinOps queries to use the same underlying storage; benchmarked catalog queries ran at 55–65 ms end-to-end and searches at two to four ms. Because Lakehouse Federation’s Postgres connector accepts static credentials while Lakebase applications use OAuth JWTs, the proof-of-concept creates a separate SCRAM-SHA-256 Postgres role for federation. This enables a single query to join Backstage resource names with Unity Catalog’s system.billing.usage without data movement, while branch billing remains visible per branch and endpoint. The operational considerations include password rotation, read-only grants, branch TTLs, and scale-to-zero endpoints that stop billing when idle.

Context

Operational ownership data in Backstage and cloud cost data in a data warehouse normally cross organizational and system boundaries, requiring ETL or coordination between platform engineering and data teams. Ephemeral database branches also need cost visibility without dynamically creating pipelines for each branch.

Approach / What changed

Lakebase separates compute per workload, and Lakehouse Federation exposes the live Postgres data alongside analytical billing data. Because Federation’s Postgres connector uses static credentials while the application uses OAuth JWTs, the setup adds a native Postgres role with SCRAM-SHA-256 authentication, read-only grants, and a separate federation connection.

Takeaways

• A FinOps query can join Backstage resource names from the live Postgres catalog with Unity Catalog’s system.billing.usage without moving the data.
• Federation currently requires a parallel static-credential authentication path: a separate SCRAM-SHA-256 Postgres role rather than the application’s OAuth JWT identity.
• Branch-level billing is exposed through branch_id and endpoint_id; short TTLs and scale-to-zero endpoints help prevent idle or orphaned branches from accumulating costs.

Related reading