DatabricksJason Reid, Ryan Blue, Daniel Weeks, Michelle LeonAnnouncement
Databricks announces a broad set of Apache Iceberg capabilities in Unity Catalog, spanning General Availability, previews, and beta releases. Managed Iceberg is GA, supporting table creation, reads, writes, optimization, governance, and sharing, while Iceberg v3 adds deletion vectors, row tracking, and VARIANT across managed, foreign, and UniForm-enabled tables. Unity Catalog also federates external catalogs, vends credentials, shares live data with Iceberg REST-compatible clients through Delta Sharing, and applies attribute-based access control during server-side scan planning for supported external engines. These capabilities are presented as a unified approach to open APIs, cross-engine governance, zero-copy sharing, and production performance without copying data. The post also outlines Iceberg v4 and a proposal for Delta 5.0 to adopt an adaptive metadata tree structure.
The open lakehouse needs catalog capabilities that preserve interoperability across engines while providing consistent governance, performance, and data sharing. The post identifies open APIs, external catalog federation, cross-engine governance, secure sharing, and continuous performance and format innovation as operational requirements.
Databricks is expanding Unity Catalog with managed and foreign Iceberg support, Iceberg REST Catalog APIs, credential vending, catalog federation, cross-engine ABAC, Delta Sharing for Iceberg clients, materialized views, and automatic optimization. It is also advancing Iceberg v3 and proposing an adaptive metadata tree structure for Iceberg v4 and Delta 5.0.
Managed Iceberg is generally available in Unity Catalog, with Predictive Optimization and Liquid Clustering for table performance.
Cross-engine ABAC evaluates column masks, row filters, and tag-based policies during server-side scan planning through Iceberg REST Catalog Scan APIs.
Iceberg v3 support includes deletion vectors, row tracking, and VARIANT across managed, foreign, and UniForm-enabled tables.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Apache Iceberg replaces Hive's partition-based metadata with a versioned catalog layer, trading operational simplicity for ACID guarantees, schema evolution, and cross-engine compatibility.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
