---
title: "How to get the most out of running dbt on Databricks"
description: "Tens of thousands of organizations use dbt, and where it runs directly affects pipeline cost, speed, and operational complexity. Running dbt on Databricks provides an open, unified platform, yet many teams underutilize its capabilities. The post details practices to reduce costs, such as applying appropriate incremental strategies and replacing full recomputes with materialized views, which delivered a 17x speedup in a benchmark. It also covers automating data layout with Liquid Clustering and Predictive Optimization, and attributing every dollar to specific models via query tags. For operational simplicity, it recommends bringing ingestion into dbt with streaming tables, standardizing metrics through Unity Catalog Semantics, and orchestrating pipelines natively with Lakeflow Jobs."
---

# How to get the most out of running dbt on Databricks

[Databricks](https://yomu.fyi/company/databricks) · Srilekha Dornadula, Luís Cabral · Oct 20, 2026

**Type:** Tutorial

## Summary

Tens of thousands of organizations use dbt, and where it runs directly affects pipeline cost, speed, and operational complexity. Running dbt on Databricks provides an open, unified platform, yet many teams underutilize its capabilities. The post details practices to reduce costs, such as applying appropriate incremental strategies and replacing full recomputes with materialized views, which delivered a 17x speedup in a benchmark. It also covers automating data layout with Liquid Clustering and Predictive Optimization, and attributing every dollar to specific models via query tags. For operational simplicity, it recommends bringing ingestion into dbt with streaming tables, standardizing metrics through Unity Catalog Semantics, and orchestrating pipelines natively with Lakeflow Jobs.

## Context

The supplied text is a blog post aimed at teams that run dbt on Databricks, offering practices to reduce pipeline costs, improve speed, and simplify operations. It is addressed to technical audiences who already use dbt.

## Approach / What changed

The blog describes practices to optimize dbt on Databricks across cost, performance, and operations. It recommends using incremental strategies, materialized views, Predictive Optimization, Liquid Clustering, query tags for cost attribution, streaming tables for ingestion, Unity Catalog Semantics for metric governance, Lakeflow Jobs for native orchestration, Declarative Automation Bundles for versioned deployment, and in-project governance via tags, grants, and row/column-level security.

## Takeaways

- Choosing the right incremental strategy per model reduces full table rebuilds; replace\_where is more efficient than merge when changes can be described with a predicate.
- Materialized views on Databricks automate incrementalization with a one-line config change and delivered a 17x speedup in a TPC-DI benchmark at the serving layer.
- Lakeflow Jobs collapse ingestion, dbt, and downstream tasks into a single native DAG with built-in scheduling, retries, and alerts, removing the need for a third-party orchestrator.

## FAQ

### Which incremental strategy should I choose?

Use append for inserts only, merge for upserts on the same key, replace\_where for narrow time windows, insert\_overwrite for whole partitions, delete+insert for key-based replacement without column-level update logic, and microbatch for time-series data with structured backfill needs.

### How do materialized views improve performance?

Materialized views automate incrementalization by tracking underlying data changes and refreshing only affected results. Switching from full recompute to materialized views at the serving layer delivered a 17x speedup in a TPC-DI benchmark.

### How do query tags attribute costs to dbt models?

Every dbt query run on Databricks is stamped with model name, dbt version, and materialization type. These tags surface in system.query.history, and joining against system.billing.usage attributes serverless warehouse spend to dimensions like model, team, or domain.

**Tags:** [Databricks](https://yomu.fyi/topic/databricks), [Data Engineering](https://yomu.fyi/topic/data-engineering), [dbt](https://yomu.fyi/topic/dbt), [SQL](https://yomu.fyi/topic/sql)

- Source: [Databricks](https://www.databricks.com/blog/how-get-most-out-running-dbt-databricks)
- Source URL: https://www.databricks.com/blog/how-get-most-out-running-dbt-databricks
- Ingested by Yomu: 2026-10-20T00:00:20.463Z

[Read original post](https://www.databricks.com/blog/how-get-most-out-running-dbt-databricks)