v1.7.4

UPDATE
This release has been pulled due to discovered connection issues with (non-serverless) SQL Warehouses. 1.7.5 with a fix will be released shortly.

For anyone waiting for materialized views / streaming tables updates, those will be coming in 1.7.6b1 (and possibly 1.7.6b2), but we had so many fixes piling up that we wanted to get this release out faster. In particular, pandas 2.2.0 release is causing havoc, so we are pinning to < 2.2.0.

What's Changed

Fixes

• Added python model specific connection handling to prevent using invalid sessions (547)
• Allow schema to be specified in testing (thanks @case-k-git!) (538)
• Fix dbt incremental_strategy behavior by fixing schema table existing check (thanks @case-k-git!) (530)
• Fixed bug that was causing streaming tables to be dropped and recreated instead of refreshed. (552)
• Fixed Hive performance regression by streamlining materialization type acquisition (557)
• Fix: Python models authentication could be overridden by a .netrc file in the user's home directory (338)
• Fix: MV/ST REST api authentication could be overriden by a .netrc file in the user's home directory (555)
• Show details in connection errors (562)
• Updated connection debugging logging and setting connection last used time on session open.(565)

Under the Hood

• Adding retries around API calls in python model submission (549)
• Upgrade to databricks-sql-connector 3.0.0 (554)
• Pinning pandas to < 2.2.0 to keep from breaking multiple tests (564)

New Contributors

• @case-k-git made their first contribution in #537
• @lennartkats-db made their first contribution in #562

Full Changelog: v1.7.3...v1.7.4

v1.6.8

What's Changed

Fixes

• Backport of fix for where we were invoking create schema or not exists when the schema already exists (leading to permission issue) (529)

Under the Hood

• Update dependency to dbt-spark 1.6.2 (thanks @ChenyuLInx!)

Full Changelog: v1.6.7...v1.6.8

v1.7.3

What's Changed

The big change in this release is that we fixed the issue where every single dbt action initiated a new connection to Databricks. We will now reuse a connection if there is a thread-local connection that matches the compute the user has selected.

This change will be most apparent if your dbt operations are very short lived, such as tests against a small table, as there is now less time spent in connection negotiation; for longer operations, the time spent in computing and transmitting the result set is more significant than the time spent on connecting.

If for some unforeseen reason this change negatively impacts performance:

a.) You can turn it off by setting the DBT_DATABRICKS_LONG_SESSIONS environment variable to false.
b.) Please file an issue so we can investigate.

Fixes

• Allow session re-use by @rcypher-databricks in #517
• Fixing list_schemas so that we stop trying to create schemas that exist by @benc-db in #529

Under the Hood

• Refactor macro tests so that we can move macros by @benc-db in #524
• Updating Python Functional Tests by @benc-db in #526
• Refactoring to align with dbt-core organization: Part I by @benc-db in #525

Full Changelog: v1.7.2...v1.7.3

1.7.2

The big news is that the ability to choose separate compute by model is now available. Until I get updated docs out, please look here for usage notes: #333 (comment)

What's Changed

• Updates to warehouse-per-model logging and tests by @rcypher-databricks in #511
• Persist docs on incremental only if changed by @benc-db in #513
• Enable access control for job runs by @benc-db (thanks @srggrs!) in #518
• View column comments can now be persisted, hive docs generate now works by @benc-db in #519

Full Changelog: v1.7.2b2...v1.7.2

1.5.7

Fixes

This release is to declare that the 1.5.x branch is not compatible with databricks-sql-connector version 3.0.0

Full Changelog: v1.5.6...v1.5.7

1.7.2b2

This is a beta release for testing the ability to specify compute on a per model basis. For full instructions on how to use this capability, for now see #333, where I will include the provisional instructions. DO NOT RELY ON THIS CAPABILITY FOR PRODUCTION WORKLOADS YET. We are looking for users to try out this feature and report any bugs they encounter.

Full Changelog: v1.7.1...v1.7.2b2

1.7.1

Under the Hood

• Revert to client-side filtering for large projects in an attempt improve performance of doc generation by @benc-db (thanks @mikealfare for the help) (503)

Full Changelog: v1.7.0...v1.7.1

1.7.0

What's Changed

This release is mostly about performance and compatibility with 1.7.x of dbt-core. Expect more to come in the coming weeks for expanding config, and config change management, for Materialized Views and Streaming Tables.

Features

• Implementing metadata freshness checks by @benc-db in #481
• Catalog fetch improvement by @benc-db in #486
• Faster dbt seeds by @benc-db (thanks @nrichards17 for the original PR!) in #498

Under the Hood

• Add node_info to SQLQuery events by @davidharting (thanks!) in #494
• 1.7.1 Compat by @benc-db (with help from @mikealfare!) in #499

New Contributors

• @davidharting made their first contribution in #494

Full Changelog: v1.7.0rc1...v1.7.0

1.6.7

Under the Hood

• Updating to dbt-spark 1.6.1

v1.7.0 RC1

What's Changed

• Getting compatibility with 1.7.0 RC by @benc-db in #479
• As part of the above change, fixed a bug with constraints where if a column had a primary key constraint and a not null constraint (a pre-req for primary key), it could fail depending on the order the constraints were run in.
• As part of the above, included support for specifying foreign key constraints using the dbt constraint expression syntax. Currently this support is restricted to single column foreign keys.

Full Changelog: v1.6.6...v1.7.0rc1