Loading post…
Supabasepaul_copplestone,bo_lu,oli_riceAnnouncement
Supabase Wrappers, a Postgres FDW framework written in Rust
Summary
Supabase Wrappers is an Alpha framework for building PostgreSQL Foreign Data Wrappers that connect databases with external systems, including third-party APIs, through SQL. It initially supports Firebase and Stripe, while wrappers for ClickHouse, BigQuery, and Airtable are under development. The framework extends pgx, a Rust Postgres extension framework, with async support for RESTful API-based wrappers and limited push-down support currently provided as a proof of concept. The post presents Wrappers as a basis for QETL, allowing teams to query, extract, transform, and load remote data through Postgres, either on demand or into local tables, while joining it with operational data. It is released outside official platform support, with potential breaking changes.
Context
PostgreSQL Foreign Data Wrappers provide a way to connect Postgres with external data sources, but the post describes the existing FDW ecosystem as difficult to assess for support and maintenance. It also motivates on-demand ETL because fully loading data can be inconvenient or unfeasible, while querying source data directly can limit reuse and increase performance or cost concerns.
Approach / What changed
Supabase extends pgx with FDW support and a framework for building wrappers in Rust. Wrappers expose external systems as Postgres foreign tables queried with SQL, support asynchronous RESTful API integrations, and include limited query push-down as a proof of concept. The QETL approach can query remote data on demand, transform it with SQL, materialize results locally, or move selected data to external warehouses.
Takeaways
The Alpha release includes Firebase and Stripe wrappers; ClickHouse, BigQuery, and Airtable wrappers are under development.
Wrappers can represent external data as Postgres foreign tables, allowing SQL queries and joins with local operational tables.
The framework is written in Rust and supports asynchronous API-based wrappers, while push-down support is currently limited and expected to receive further focus.
Source and freshness
- Source
- supabase.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.