Loading post…
Supabasericcardo_busettiAnnouncement
Supabase Pipelines is now in Public Alpha
Summary
Supabase Pipelines has moved into public alpha as a service for replicating Supabase Postgres data to external analytical systems in near real time. The release adds automatic application of supported schema changes, faster initial synchronization through parallelism, and a destination request form for ClickHouse, Snowflake, and DuckLake; Google BigQuery is the first destination available to everyone in public alpha. Powered by the open-source Rust-based Supabase ETL, Pipelines uses Postgres logical replication to copy existing tables, stream inserts, updates, deletes, and truncates, and resume from the last acknowledged position after interruptions. It provides at-least-once delivery and managed pipeline controls, while remaining an alpha product with ongoing work on throughput, batching, backpressure, broader schema support, and additional destinations.
Context
Supabase Pipelines is intended to move production Postgres data into systems built for analytics, avoiding the load that large scans, aggregations, dashboards, reports, notebooks, and downstream systems can place on the transactional database serving an application.
Approach / What changed
Pipelines uses Supabase ETL, an open-source Rust change-data-capture pipeline, and Postgres logical replication. It performs an initial sync of selected tables with parallel workers, then batches ongoing changes to the destination, applies supported schema changes, and resumes from the last acknowledged position after interruptions or restarts.
Takeaways
Supported schema changes currently include adding, removing, or renaming columns and changing column nullability or defaults; unsupported changes may still need manual handling.
Initial syncs can run in parallel across tables and within a table, then switch to ongoing replication that batches changes from the replication slot.
Public alpha pricing is $0.053 per hour per configured pipeline, $0.60 per GB for initial sync data, and $3 per GB for ongoing replication; pipelines are billed while paused or inactive.
Source and freshness
- Source
- supabase.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.