Loading…
Faster Queries and New Capabilities with the Open-Source Databricks JDBC Driver
Toussaint Webb, Gopal Lal, Kaitlin Baumgardner
- Source
- Databricks
- Published
- Added to Yomu
Summary
The open-source Databricks JDBC driver 3.x and later releases address connectivity needs affecting dashboards, spreadsheets, applications, and other data workflows. Compared with the legacy 2.x driver, they deliver up to 30% faster retrieval of large query results while preserving backward compatibility. Architectural changes add full Arrow-based transfer for JDK 16+, a first-class asynchronous execution API, stream-based ingestion into Databricks Volumes, and integration with the Statement Execution API. The driver also supports stored procedures, multi-statement transactions, Unity Catalog metric views, query tags, geospatial and complex data types, plus built-in client telemetry for query latency, metrics, and errors. Its open-source ownership and Databricks maintenance are presented as enabling faster fixes, new features, transparency, contributions, and tighter platform integration.
Context
Modern dashboards, spreadsheet analysis, applications, and other workflows depend on fast, reliable connectivity to data, while retrieving large query results can become a major performance bottleneck. The release also addresses the need for newer Databricks capabilities, richer SQL workflows, and better visibility into query and connection behavior.
Approach / What changed
Databricks enhanced its open-source JDBC driver in releases 3.x and above. Changes include Arrow-based transfer for JDK 16+, asynchronous statement execution, stream-based ingestion into Databricks Volumes, Statement Execution API integration, expanded SQL and data-type support, built-in client telemetry, and continued open-source ownership and maintenance by Databricks.
Takeaways
- The 3.x-and-above driver delivers up to 30% faster large-result retrieval than the legacy 2.x JDBC driver, with backward compatibility maintained during migration.
- The driver supports Arrow transfer on JDK 16+, asynchronous query submission, direct Statement Execution API integration, and streaming bulk ingestion into Databricks Volumes without local staging.
- New functionality covers stored procedures, multi-statement transactions, Unity Catalog metric views, query tags, geospatial data, complex types, and telemetry for latency, metrics, and errors.