Loading post…
DatabricksCan Efeoglu, Sachin Thakur, Richard Tomlinson, Justin Talbot, Daniel Rosenthal, Jasmeet JaggiAnnouncement
Announcing General Availability and Open Sourcing of Unity Catalog Business Semantics
Summary
Databricks announces the general availability of Unity Catalog Business Semantics, a unified semantic foundation for consistent business definitions across BI dashboards, notebooks, SQL applications, and AI agents. Built into Unity Catalog, it applies existing governance, security, lineage, and access controls to centrally defined Metric Views, whose metadata includes display names, formats, and synonyms. Metric Views are authored declaratively in SQL, compiled and executed deterministically at query time, and can use materializations with automatic pre-aggregation, incremental refresh, intelligent query rewriting, and transparent routing; materialization remains in Preview. The core Metric View implementation is being open sourced in Apache Spark OSS for an upcoming release, with Unity Catalog OSS v0.5 support coming soon, while a point-and-click authoring UI is in Public Preview.
Context
The announcement addresses metric drift and conflicting reports caused by business definitions being fragmented across BI tools, dashboards, models, and spreadsheets. It positions business semantics as a governed, reusable layer at the data foundation rather than a presentation-tier capability, with portability across analytics and AI surfaces.
Approach / What changed
Unity Catalog Business Semantics centralizes business definitions in Metric Views governed through Unity Catalog. Definitions use SQL and semantic metadata, are exposed through SQL and APIs, and can be consumed across dashboards, notebooks, applications, AI agents, and connected third-party tools. Materializations provide pre-aggregation, incremental refresh, query rewriting, and transparent routing, while the core implementation is being open sourced in Apache Spark OSS.
Takeaways
Metric Views centralize KPI definitions, data mappings, measures, dimensions, and metadata such as display names, formats, and synonyms in Unity Catalog.
Materialization is in Preview and maintains optimized pre-aggregated results through incremental refresh, with query rewriting and routing intended to improve performance without changing how users query metrics.
The core Metric View implementation is being open sourced in Apache Spark OSS, targeting an upcoming Apache Spark release; Unity Catalog OSS v0.5 support is coming soon.
FAQ
How are Metric Views kept consistent across different consumers?
Metric Views are defined declaratively in SQL and governed centrally in Unity Catalog. The engine compiles and executes the underlying SQL deterministically at query time, so consumers receive the same result from the same definition across supported surfaces.
How does materialization improve Metric View performance?
Materializations maintain optimized, pre-aggregated results through incremental updates. At query time, the engine can rewrite queries and route requests to the best available materialization, while users continue querying metrics in the same way.
Can business semantics be used outside proprietary BI tools?
Yes. The definitions are accessed through SQL and APIs, stored in an open format, and designed for dashboards, notebooks, applications, AI agents, and third-party tools connected to Databricks. The core Metric View implementation is also being open sourced in Apache Spark OSS.
Source and freshness
- Source
- databricks.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.