Loading…
Introducing Feature Views
Nick Joung, Ian Ackerman, Julia Powell
- Source
- Databricks
- Published
- Added to Yomu
Summary
Feature Views are presented as a framework for creating managed feature pipelines in Databricks, addressing duplicated logic, training/serving skew, backfills, online-store plumbing, and governance for real-time ML. A Feature View defines the source, entity, time-series column, and computation once; Databricks uses it to generate point-in-time-accurate historical data, then materializes pipelines for batch or streaming production use. Registered in Unity Catalog, materialized features receive lineage and access controls, while MLflow records model dependencies and Model Serving retrieves required features without custom lookup code. For Kafka-sourced streaming features, Spark Realtime Mode, Lakebase, and Model Serving support rolling-window updates with reported end-to-end p99 latency of 200ms from event to online availability. The announcement says Feature Views are in Public Preview and can be developed through the Feature Engineering Client SDK and Genie Code, with streaming materialization requiring an Enterprise-tier workspace in a region supporting Lakebase.
Context
Teams often face duplicated feature logic between real-time and historical training, training/serving skew, difficult backfills, complex online-store infrastructure, and governance overhead when deploying fresh signals for use cases such as fraud detection, personalization, and recommendations.
Approach / What changed
Feature Views let users define a feature’s source, entity, time-series column, and computation once, then generate historical training data and managed batch or streaming pipelines. Databricks handles materialization, online and offline stores, lineage, model dependencies, and serving integration through Unity Catalog, MLflow, and Model Serving.
Takeaways
- A single Feature View definition is used for both point-in-time-accurate training data and online inference values, reducing the gap between training and serving computations.
- Kafka-sourced streaming features use RollingWindow computations with millisecond timestamp resolution; the described system reports 200ms end-to-end p99 latency from event arrival to online availability.
- Feature Views are first-class Unity Catalog objects, with access control and lineage; MLflow records feature dependencies so Model Serving can retrieve required features without custom lookup code.