DatabricksRyan Blue, Marco KrollExplainer
The post explains how the REGISTER and UNREGISTER APIs in the Apache Iceberg REST catalog specification enable table portability between catalogs without copying or moving data files. REGISTER attaches an existing table to a new catalog by providing the metadata location, but using it alone creates a split-brain scenario where two catalogs both think they own the table, leading to inconsistent writes and data loss. The UNREGISTER endpoint, contributed to the specification, cleanly removes the table from the old catalog and returns the latest metadata pointer needed by the next catalog. The handoff process combines unregister, receiving the location, and register to ensure exactly one managing catalog at all times. Unity Catalog supports these endpoints in private preview, allowing organizations to move tables between catalogs as their architecture evolves.
The post describes the REGISTER and UNREGISTER endpoints in the Apache Iceberg REST catalog specification, contributed by the Unity Catalog team, to enable table portability between catalogs without data copying.
The REGISTER endpoint attaches an existing table to a new catalog by providing the metadata location. The UNREGISTER endpoint removes the table from the old catalog and returns the latest metadata pointer, enabling a clean handoff to the new catalog without data file modification.
REGISTER alone creates a split-brain scenario where two catalogs both think they own the table, leading to inconsistent writes and data loss.
UNREGISTER removes the table from the old catalog without deleting data files and returns the metadata location needed by the next catalog.
The handoff process involves three steps: unregister from old catalog, receive the metadata location, and register in the new catalog, ensuring a single managing catalog.
Using REGISTER alone creates a split-brain scenario where two catalogs both think they own the table, leading to inconsistent writes and silent data loss because the catalogs do not communicate.
UNREGISTER removes the table entry from the old catalog without touching data files and returns the latest metadata pointer, allowing the new catalog to assume commit coordination and preventing split-brain.
The handoff involves three steps: UNREGISTER the table from the old catalog, receive the metadata location, and REGISTER it in the new catalog, ensuring exactly one managing catalog at all times.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
A tradeoff view of blockchain: a shared ledger buys verifiable settlement between parties that do not share an operator, and charges throughput, latency, and governance in return.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
