Introduce `SeaDatabricksClient` (Session Implementation) #582

varun-edachali-dbx · 2025-06-03T08:27:36Z

What type of PR is this?

Feature

Description

Introduce SEA Backend client as an alternative to the existing Thrift backend
implement open_session() and close_session() functionality.

How is this tested?

Unit tests
E2E Tests
Manually - the test script examples/experimental/sea_connector_test.py in the sea-migration branch contains a simple function that creates and closes a connection using the SEA mode, effectively opening and closing a session using the SEA client. This was run, and verified to work.
N/A

The coverage of the newly introduced backend/sea directory by the unit tests is as below:

Module	Statements	Missing	Coverage	Notes
`backend.py`	96	1	99%	Line 6: `from databricks.sql.client import Cursor` (TYPE_CHECKING import)
`models/__init__.py`	3	0	100%	Fully covered
`models/requests.py`	23	0	100%	Fully covered
`models/responses.py`	8	0	100%	Fully covered
`utils/constants.py`	2	0	100%	Fully covered
`utils/http_client.py`	76	63	17%	Missing coverage in: - Lines 44-80: SSL configuration in `__init__` - Lines 86-88: `_get_auth_headers` method - Lines 92-99: `_get_call` method - Lines 124-186: Error handling in `_make_request` method

Note: I intentionally did not add unit tests for the HTTP client because I do not expect it to change and it is for the short term. It has been indirectly tested as a part of the manual tests.

Related Tickets & Documents

https://docs.google.com/document/d/1Y-eXLhNqqhrMVGnOlG8sdFrCxBTN1GdQvuKG4IfHmo0/edit?usp=sharing
https://databricks.atlassian.net/browse/PECOBLR-482?atlOrigin=eyJpIjoiODdkMGQ5NjZhMjY5NDRmZWEwZmIyNGEyNzFlNGQxNzEiLCJwIjoiaiJ9

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

samikshya-db · 2025-06-06T11:03:03Z

Have we tested this end to end locally? If yes, let's add that detail in the PR description

jayantsing-db

Minor comments. Rest LGTM

src/databricks/sql/backend/models/__init__.py

jayantsing-db · 2025-06-07T15:12:45Z

src/databricks/sql/backend/sea_backend.py

+        """
+
+        warehouse_pattern = re.compile(r".*/warehouses/(.+)")
+        endpoint_pattern = re.compile(r".*/endpoints/(.+)")


do we have endpoints now? i don't think so

The JDBC driver supports it.

jayantsing-db · 2025-06-07T15:17:29Z

src/databricks/sql/backend/sea_backend.py

+        Opens a new session with the Databricks SQL service using SEA.
+
+        Args:
+            session_configuration: Optional dictionary of configuration parameters for the session


right now only a select spark configs are allowed in DBSQL as session configs: https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-parameters

please document this appropriately. also, please test if server ignores any other arbitrary session config. if not, please include client-side checks/filtering

Thanks, I added a link to the above doc in the doc-string.

I tested passing some arbitrary (un-supported) parameters in the session config and I did not get any warning or error from the server, so the server seems to ignore irrelevant params.

To be clear, this means we need not include any client side filtering, right?

src/databricks/sql/backend/sea_backend.py

tests/unit/test_sea_backend.py

tests/unit/test_session.py

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

samikshya-db

Minor comments

src/databricks/sql/backend/sea/backend.py

…ver endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 08:27 — with GitHub Actions Inactive

varun-edachali-dbx had a problem deploying to azure-prod June 3, 2025 08:30 — with GitHub Actions Failure

varun-edachali-dbx mentioned this pull request Jun 3, 2025

Introduce SeaDatabricksClient (Session Implementation) #577

Closed

5 tasks

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 08:33 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 08:52 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 09:21 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 09:37 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 09:39 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 3, 2025 16:08 — with GitHub Actions Inactive

varun-edachali-dbx had a problem deploying to azure-prod June 4, 2025 05:44 — with GitHub Actions Failure

varun-edachali-dbx had a problem deploying to azure-prod June 4, 2025 05:57 — with GitHub Actions Failure

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 05:59 — with GitHub Actions Inactive

varun-edachali-dbx force-pushed the sessions-sea branch from b97b9be to 6c6ec28 Compare June 4, 2025 05:59

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 05:59 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 06:19 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 06:22 — with GitHub Actions Inactive

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 06:25 — with GitHub Actions Inactive

varun-edachali-dbx marked this pull request as ready for review June 4, 2025 06:28

varun-edachali-dbx requested review from deeksha-db, samikshya-db, jprakash-db, jackyhu-db, madhav-db, gopalldb, jayantsing-db, vikrantpuppala and shivam2680 as code owners June 4, 2025 06:28

varun-edachali-dbx temporarily deployed to azure-prod June 4, 2025 06:45 — with GitHub Actions Inactive

varun-edachali-dbx changed the title ~~Introduce SeaDatabricksClient (Session Implementation)~~ Introduce SeaDatabricksClient (Session Implementation) Jun 4, 2025

varun-edachali-dbx temporarily deployed to azure-prod June 5, 2025 04:34 — with GitHub Actions Inactive

databricks deleted a comment from github-actions bot Jun 5, 2025

Merge branch 'sea-migration' into sessions-sea

638e1df

varun-edachali-dbx temporarily deployed to azure-prod June 5, 2025 05:03 — with GitHub Actions Inactive

databricks deleted a comment from github-actions bot Jun 5, 2025

varun-edachali-dbx temporarily deployed to azure-prod June 5, 2025 05:46 — with GitHub Actions Inactive

redundant imports

ed4931e

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx force-pushed the sessions-sea branch from 2e3d5d0 to ed4931e Compare June 5, 2025 05:47

varun-edachali-dbx temporarily deployed to azure-prod June 5, 2025 05:47 — with GitHub Actions Inactive

databricks deleted a comment from github-actions bot Jun 5, 2025

jayantsing-db approved these changes Jun 7, 2025

View reviewed changes

move sea backend and models into separate sea/ dir

4ff64ed

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx temporarily deployed to azure-prod June 7, 2025 16:41 — with GitHub Actions Inactive

move http client into separate sea/ dir

9aebea2

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx temporarily deployed to azure-prod June 7, 2025 16:53 — with GitHub Actions Inactive

varun-edachali-dbx added 2 commits June 7, 2025 17:35

change commands to include ones in docs

a05f1fd

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

add link to sql-ref-parameters for session-confs

46104e2

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx temporarily deployed to azure-prod June 7, 2025 17:49 — with GitHub Actions Inactive

samikshya-db approved these changes Jun 8, 2025

View reviewed changes

src/databricks/sql/backend/sea/backend.py Outdated Show resolved Hide resolved

src/databricks/sql/backend/sea/backend.py Show resolved Hide resolved

src/databricks/sql/backend/sea/backend.py Show resolved Hide resolved

varun-edachali-dbx added 2 commits June 8, 2025 16:03

add client side filtering for session confs, add note on warehouses o…

390c1e7

…ver endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

test unimplemented methods and max_download_threads prop

86ee56f

Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com>

varun-edachali-dbx temporarily deployed to azure-prod June 9, 2025 01:41 — with GitHub Actions Inactive

varun-edachali-dbx merged commit 0887bc1 into sea-migration Jun 9, 2025
23 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Introduce `SeaDatabricksClient` (Session Implementation) #582

Introduce `SeaDatabricksClient` (Session Implementation) #582

Uh oh!

varun-edachali-dbx commented Jun 3, 2025 •

edited

Loading

Uh oh!

samikshya-db commented Jun 6, 2025

Uh oh!

jayantsing-db left a comment

Uh oh!

Uh oh!

jayantsing-db Jun 7, 2025

Uh oh!

varun-edachali-dbx Jun 7, 2025 •

edited

Loading

Uh oh!

jayantsing-db Jun 7, 2025

Uh oh!

varun-edachali-dbx Jun 7, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samikshya-db left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Introduce SeaDatabricksClient (Session Implementation) #582

Introduce SeaDatabricksClient (Session Implementation) #582

Uh oh!

Conversation

varun-edachali-dbx commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

Description

How is this tested?

Related Tickets & Documents

Uh oh!

samikshya-db commented Jun 6, 2025

Uh oh!

jayantsing-db left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jayantsing-db Jun 7, 2025

Choose a reason for hiding this comment

Uh oh!

varun-edachali-dbx Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jayantsing-db Jun 7, 2025

Choose a reason for hiding this comment

Uh oh!

varun-edachali-dbx Jun 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samikshya-db left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Introduce `SeaDatabricksClient` (Session Implementation) #582

Introduce `SeaDatabricksClient` (Session Implementation) #582

varun-edachali-dbx commented Jun 3, 2025 •

edited

Loading

varun-edachali-dbx Jun 7, 2025 •

edited

Loading

varun-edachali-dbx Jun 7, 2025 •

edited

Loading