Query Tags: The Context Your Warehouse Queries Have Been Missing

Summary

Databricks SQL Query Tags address a visibility gap in warehouse workloads: built-in logs identify users, warehouses, and tools, but not dashboards, projects, teams, or cost centers. Query Tags, now in Public Preview, attach multiple custom key-value pairs to each SQL execution, carry them into the Query History System Table, and expose them in the Query Profile UI. Partner integrations can automatically tag dbt models, while Power BI and Tableau support connection-level tags, and APIs and connectors support connection- or statement-level metadata. Users can also issue SET QUERY_TAGS in SQL Editor, notebooks, dashboards, or alerts to label subsequent session statements. These tags enable SQL or Genie queries for cost allocation, regression analysis, workload filtering, and environment comparisons; future plans include broader connector and workload support plus Query History search.

Context

Databricks SQL logs who ran a query, which warehouse handled it, and which tool submitted it, but this does not identify the originating dashboard, project, team, cost center, application, or dbt model. That limits workload attribution, cost allocation, and investigation of slow or regressed queries.

Approach / What changed

Query Tags attach custom key-value pairs to SQL executions and record them in the Query History System Table. Tags can be injected automatically by partner tools, set at connection or statement level through APIs and connectors, or applied with SET QUERY_TAGS for subsequent session statements; the resulting metadata can be queried with SQL or Genie.

Takeaways

• dbt automatically adds tags for model name, core version, adapter version, and materialization type, enabling model-level workload and regression analysis.
• Power BI and Tableau support connection-level Query Tags, while the SQL Statement Execution API and Python Connector support statement-level tagging; the Python Connector also supports connection-level tags.
• The query_tags column in the Query History System Table supports grouping, filtering, cost chargeback, pipeline monitoring, workbook attribution, and dev-versus-production comparisons.

Related reading