Loading…
Unlock seamless and cost-effective marketing campaigns with Lakebase
Thomas Nguyen
- Source
- Databricks
- Published
- Added to Yomu
Summary
Retail marketing teams often store customer segments in OLTP databases, creating underused capacity between campaigns and synchronization work whenever new segments are requested. Lakebase, Databricks’ implementation of a lakebase architecture, separates storage from compute and uses elastic, serverless Postgres that can scale to zero when idle and up for bursty campaign traffic. The integration with the Lakehouse uses managed Synced Tables for customer segments and Lakehouse Sync for continuous CDC-based replication of operational data into Unity Catalog Delta tables. The SAP Engagement Cloud example configures a Lakebase Autoscaling project, native Postgres credentials, an ISRG Root X1 certificate, and snapshot synchronization for datasets with more than 10% updated. The post concludes that this design lowers idle and sizing costs, reduces pipeline maintenance, and supports low-latency point lookups, while warning that Lakebase is not optimized for large scans or classic OLAP.
Context
Retail companies face underused OLTP database capacity outside marketing campaigns and an operational burden when data teams create and maintain Lakehouse-to-OLTP synchronization pipelines for changing customer segments. Campaign traffic is bursty, with periodic query spikes separated by extended idle periods.
Approach / What changed
The integration uses Databricks Lakebase Postgres with separated storage and compute, elastic serverless autoscaling, and managed synchronization with the Lakehouse. The example connects SAP Engagement Cloud through a Postgres role and CA certificate, synchronizes customer segments with Synced Tables, and sends operational data back through continuous CDC-based Lakehouse Sync.
Takeaways
- Lakebase can scale compute to zero during extended lows and up to a configured 16 CU maximum for campaign spikes, aligning costs with usage.
- For customer segments where more than 10% of the data changes, snapshot synchronization is recommended and is described as delivering 10x better performance than triggered mode.
- Lakebase is optimized for high-concurrency point lookups and short OLTP queries, not large scans or traditional OLAP workloads.