Update databricks-sdk requirement from <0.40,>=0.39 to >=0.39,<0.41 (#3456)

Updates the requirements on
[databricks-sdk](https://github.com/databricks/databricks-sdk-py) to
permit the latest version.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-py/releases">databricks-sdk's
releases</a>.</em></p>
<blockquote>
<h2>v0.40.0</h2>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/account/account_federation_policy.html">a.account_federation_policy</a>
account-level service and <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/account/service_principal_federation_policy.html">a.service_principal_federation_policy</a>
account-level service.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterAttributes</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterDetails</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterSpec</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.CreateCluster</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.EditCluster</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.UpdateClusterResource</code>.</li>
<li>Added <code>update_parameter_syntax</code> field for
<code>databricks.sdk.service.dashboards.MigrateDashboardRequest</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.RunTask</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.SubmitTask</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.Task</code>.</li>
<li>Changed <code>days_of_week</code> field for
<code>databricks.sdk.service.pipelines.RestartWindow</code> to type
<code>databricks.sdk.service.pipelines.RestartWindowDaysOfWeekList</code>
dataclass.</li>
</ul>
<p>OpenAPI SHA: a6a317df8327c9b1e5cb59a03a42ffa2aabeef6d, Date:
2024-12-16</p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/databricks/databricks-sdk-py/blob/main/CHANGELOG.md">databricks-sdk's
changelog</a>.</em></p>
<blockquote>
<h2>[Release] Release v0.40.0</h2>
<h3>API Changes:</h3>
<ul>
<li>Added <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/account/account_federation_policy.html">a.account_federation_policy</a>
account-level service and <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/account/service_principal_federation_policy.html">a.service_principal_federation_policy</a>
account-level service.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterAttributes</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterDetails</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.ClusterSpec</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.CreateCluster</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.EditCluster</code>.</li>
<li>Added <code>is_single_node</code>, <code>kind</code> and
<code>use_ml_runtime</code> fields for
<code>databricks.sdk.service.compute.UpdateClusterResource</code>.</li>
<li>Added <code>update_parameter_syntax</code> field for
<code>databricks.sdk.service.dashboards.MigrateDashboardRequest</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.RunTask</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.SubmitTask</code>.</li>
<li>Added <code>clean_rooms_notebook_task</code> field for
<code>databricks.sdk.service.jobs.Task</code>.</li>
<li>Changed <code>days_of_week</code> field for
<code>databricks.sdk.service.pipelines.RestartWindow</code> to type
<code>databricks.sdk.service.pipelines.RestartWindowDaysOfWeekList</code>
dataclass.</li>
</ul>
<p>OpenAPI SHA: a6a317df8327c9b1e5cb59a03a42ffa2aabeef6d, Date:
2024-12-16</p>
<h2>[Release] Release v0.39.0</h2>
<h3>Bug Fixes</h3>
<ul>
<li>Update Changelog file (<a
href="https://redirect.github.com/databricks/databricks-sdk-py/pull/830">#830</a>).</li>
</ul>
<h3>Internal Changes</h3>
<ul>
<li>Fix a couple of typos in open_ai_client.py (<a
href="https://redirect.github.com/databricks/databricks-sdk-py/pull/829">#829</a>).</li>
<li>Update SDK to OpenAPI spec (<a
href="https://redirect.github.com/databricks/databricks-sdk-py/pull/834">#834</a>).</li>
</ul>
<h3>API Changes:</h3>
<ul>
<li>Added <code>databricks.sdk.service.cleanrooms</code> package.</li>
<li>Added <code>delete()</code> method for <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/aibi_dashboard_embedding_access_policy.html">w.aibi_dashboard_embedding_access_policy</a>
workspace-level service.</li>
<li>Added <code>delete()</code> method for <a
href="https://databricks-sdk-py.readthedocs.io/en/latest/workspace/settings/aibi_dashboard_embedding_approved_domains.html">w.aibi_dashboard_embedding_approved_domains</a>
workspace-level service.</li>
<li>Added <code>databricks_gcp_service_account</code> field for
<code>databricks.sdk.service.catalog.CreateCredentialRequest</code>.</li>
<li>Added <code>databricks_gcp_service_account</code> field for
<code>databricks.sdk.service.catalog.CredentialInfo</code>.</li>
<li>Added <code>gcp_options</code> field for
<code>databricks.sdk.service.catalog.GenerateTemporaryServiceCredentialRequest</code>.</li>
<li>Added <code>databricks_gcp_service_account</code> field for
<code>databricks.sdk.service.catalog.UpdateCredentialRequest</code>.</li>
<li>Added <code>cached_query_schema</code> field for
<code>databricks.sdk.service.dashboards.QueryAttachment</code>.</li>
<li>Added .</li>
<li>Removed <code>gcp_service_account_key</code> field for
<code>databricks.sdk.service.catalog.CreateCredentialRequest</code>.</li>
</ul>
<p>OpenAPI SHA: 7016dcbf2e011459416cf408ce21143bcc4b3a25, Date:
2024-12-05</p>
<h2>[Release] Release v0.38.0</h2>
<h3>New Features and Improvements</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/databricks/databricks-sdk-py/commit/8975d07c0cd5f3e15ce6aabf8bc905b367740b5f"><code>8975d07</code></a>
[Release] Release v0.40.0 (<a
href="https://redirect.github.com/databricks/databricks-sdk-py/issues/840">#840</a>)</li>
<li>See full diff in <a
href="https://github.com/databricks/databricks-sdk-py/compare/v0.39.0...v0.40.0">compare
view</a></li>
</ul>
</details>
<br />

<details>
<summary>Most Recent Ignore Conditions Applied to This Pull
Request</summary>

| Dependency Name | Ignore Conditions |
| --- | --- |
| databricks-sdk | [>= 0.25.a, < 0.26] |
</details>


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cor Zuurmond <jczuurmond@protonmail.com>
Co-authored-by: Guenia Izquierdo Delgado <guenia.izquierdo@databricks.com>

Author: 3 people

pyproject.toml

@@ -44,7 +44,7 @@ classifiers = [
     "Topic :: Utilities",
 ]
 
-dependencies = ["databricks-sdk>=0.39,<0.40",
+dependencies = ["databricks-sdk>=0.40,<0.41",
                 "databricks-labs-lsql>=0.14.0,<0.15",
                 "databricks-labs-blueprint>=0.9.1,<0.10",
                 "PyYAML>=6.0.0,<7.0.0",