Skip to content

Commit 141a004

Browse files
SEA: Execution Phase (#645)
* Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce `SeaDatabricksClient` (Session Implementation) (#582) * [squashed from prev branch] introduce sea client with session open and close functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidental changes to workflows (merge artifacts) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass test_input to get_protocol_version instead of session_id to maintain previous API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black + line gaps after multi-line pydocs) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use factory for backend instantiation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce models for requests and responses Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove http client and test script to prevent diff from showing up post http-client merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce Sea HTTP Client and test script (#583) * introduce http client (temp) and sea test file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce verbosity Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce redundancy, params and data separate Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type issues Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce repetition in request calls Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary elifs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add newline at EOF Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * CustomHttpClient -> SeaHttpClient Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment in backend client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * regex for warehouse_id instead of .split, remove excess imports and behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [nit] reduce nested code Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line gap after multi-line pydoc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move sea backend and models into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move http client into separate sea/ dir Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change commands to include ones in docs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add link to sql-ref-parameters for session-confs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add client side filtering for session confs, add note on warehouses over endoints Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test unimplemented methods and max_download_threads prop Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Normalise Execution Response (clean backend interfaces) (#587) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce models for `SeaDatabricksClient` (#595) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Introduce preliminary SEA Result Set (#588) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid_to_hex_id import to utils Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff in guid utils import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move arrow_schema_bytes back into ExecuteResult Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * maintain log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary assignment Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary tuple response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-ncessary verbose mocking Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move Queue construction to ResultSert Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description to List[Tuple] Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * frmatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove explicit tuple conversion) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows from ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary has_more_rows aclc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * default has_more_rows to True Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * return has_more_rows from ExecResponse conversion during GetRespMetadata Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary replacement Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better mocked backend naming Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove has_more_rows test in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce replacement of original has_more_rows read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_use_arrow_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * call correct method in test_fall_back_to_hive_schema Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce result response read test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess fetch_results mocks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more minimal changes to thrift_backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move back to old table types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove outdated arrow_schema_bytes return Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align SeaResultSet with new structure Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct sea res set tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rephrase model docstrings to explicitly denote that they are representations and not used over the wire Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * switch docstring format to align with Connection class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * has_more_rows -> is_direct_results Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix type errors with arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * spaces after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate queue init (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff (remove newlines) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 anyway Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "remove un-necessary changes" This reverts commit a70a6cee277db44d6951604e890f91cae9f92f32. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * b"" -> None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid ExecuteResponse import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Separate Session related functionality from Connection class (#571) * decouple session class from existing Connection ensure maintenance of current APIs of Connection while delegating responsibility Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add open property to Connection to ensure maintenance of existing API Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests to address ThriftBackend through session instead of through Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: move session specific tests from test_client to test_session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) as in CONTRIBUTING.md Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use connection open property instead of long chain through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * trigger integration workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: ensure open attribute of Connection never fails in case the openSession takes long, the initialisation of the session will not complete immediately. This could make the session attribute inaccessible. If the Connection is deleted in this time, the open() check will throw because the session attribute does not exist. Thus, we default to the Connection being closed in this case. This was not an issue before because open was a direct attribute of the Connection class. Caught in the integration tests. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: de-complicate earlier connection open logic earlier, one of the integration tests was failing because 'session was not an attribute of Connection'. This is likely tied to a local configuration issue related to unittest that was causing an error in the test suite itself. The tests are now passing without checking for the session attribute. https://github.com/databricks/databricks-sql-python/pull/567/commits/c676f9b0281cc3e4fe9c6d8216cc62fc75eade3b Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "fix: de-complicate earlier connection open logic" This reverts commit d6b1b196c98a6e9d8e593a88c34bbde010519ef4. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [empty commit] attempt to trigger ci e2e workflow Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Update CODEOWNERS (#562) new codeowners Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Enhance Cursor close handling and context manager exception management to prevent server side resource leaks (#554) * Enhance Cursor close handling and context manager exception management * tests * fmt * Fix Cursor.close() to properly handle CursorAlreadyClosedError * Remove specific test message from Cursor.close() error handling * Improve error handling in connection and cursor context managers to ensure proper closure during exceptions, including KeyboardInterrupt. Add tests for nested cursor management and verify operation closure on server-side errors. * add * add Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * PECOBLR-86 improve logging on python driver (#556) * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * PECOBLR-86 Improve logging for debug level Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * fixed format Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * changed debug to error logs Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * used lazy logging Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> --------- Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit dbb2ec52306b91072a2ee842270c7113aece9aff, reversing changes made to 7192f117279d4f0adcbafcdf2238c18663324515. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Reapply "Merge remote-tracking branch 'upstream/sea-migration' into decouple-session" This reverts commit bdb83817f49e1d88a01679b11da8e55e8e80b42f. Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: separate session opening logic from instantiation ensures correctness of self.session.open call in Connection Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: use is_open attribute to denote session availability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: access thrift backend through session Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: use get_handle() instead of private session attribute in client Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: remove accidentally removed assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> Signed-off-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> Co-authored-by: Jothi Prakash <jothi.prakash@databricks.com> Co-authored-by: Madhav Sainanee <madhav.sainanee@databricks.com> Co-authored-by: Sai Shree Pradhan <saishree.pradhan@databricks.com> * Introduce Backend Interface (DatabricksClient) (#573) NOTE: the `test_complex_types` e2e test was not working at the time of this merge. The test must be triggered when the test is back up and running as intended. * remove excess logs, assertions, instantiations large merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) + remove excess log (merge artifact) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce __str__ methods for CommandId and SessionId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * docstrings for DatabricksClient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing of Cursor and ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove utility functions from backend interface, fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rename info to properties Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move guid to hex id to new utils module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move staging allowed local path to connection props Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong return type for execute_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * skip auth, error handling in databricksclient interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring + line width Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * get_id -> get_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: to_hex_id -> to_hex_guid Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement ResultSet Abstraction (backend interfaces for fetch phase) (#574) * ensure backend client returns a ResultSet type in backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * newline for cleanliness Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix circular import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * to_hex_id -> get_hex_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * better comment on protocol version getter Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stricter typing for cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct typing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * correct tests and merge artifacts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally modified workflow files remnants of old merge Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * chore: remove accidentally modified workflow files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back accidentally removed docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean up docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * log hex Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary _replace call Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add __str__ for CommandId Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * take TOpenSessionResp in get_protocol_version to maintain existing interface Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * active_op_handle -> active_mmand_id Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure None returned for close_command Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * account for ResultSet return in new pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pydoc for types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move common state to ResultSet aprent Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * stronger typing in resultSet behaviour Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant patch in test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add has_been_closed_server_side assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundancies in tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more robust close check Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised state in e2e test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify corrected test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add line gaps after multi-line pydocs for consistency Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use normalised CommandState type in ExecuteResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary initialisation assertions Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary line break s Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more un-necessary line breaks Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * constrain diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff of test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use pytest-like assertions for test_closing_connection_closes_commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure command_id is not None Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * line breaks after multi-line pyfocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * ensure non null operationHandle for commandId creation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use command_id methods instead of explicit guid_to_hex_id conversion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary artifacts in test_session, add back assertion Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Implement SeaDatabricksClient (Complete Execution Spec) (#590) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comments Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove fetch phase methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code repetititon + introduce gaps after multi line pydocs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move description extraction to helper func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add more unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * streamline unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * test getting the list of allowed configurations Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * house constants in enums for readability and immutability Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add note on hybrid disposition Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove redundant note on arrow_schema_bytes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove invalid import Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for manifest in _extract_description Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary column skipping Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove parsing in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix: convert sea statement id to CommandId type Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make polling interval a separate constant Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align state checking with Thrift implementation Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * update unit tests according to changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add unit tests for added methods Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add spec to description extraction docstring, add strong typing to params Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add strong typing for backend parameters arg Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> --------- Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add from __future__ import annotations to remove string literals around forward refs, remove some unused imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move docstring of DatabricksClient within class Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * move ThriftResultSet import to top of file Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * make backend/utils __init__ file empty Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use from __future__ import annotations to remove string literals around Cursor Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * use lazy logging Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * replace getters with property tag Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * set active_command_id to None, not active_op_handle Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * align test_session with pytest instead of unittest Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove duplicate test, correct active_command_id attribute Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * SeaDatabricksClient: Add Metadata Commands (#593) * [squash from exec-sea] bring over execution phase changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess test Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remvoe exec func in sea backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess files Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess models Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess sea backend tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * cleanup Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * re-introduce get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove SeaResultSet Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * clean imports and attributes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * pass CommandId to ExecResp Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove changes in types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add back essential types (ExecResponse, from_sea_state) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix fetch types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce diff by maintaining logs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * fix int test types Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from exec-sea] init execution func Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove ResultSetFilter functionality Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * even more irrelevant changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove sea response as init option Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * exec test example scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * [squashed from sea-exec] merge sea stuffs Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess removed docstring Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes in backend Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess imports Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove accidentally removed _get_schema_desc Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove unnecessary init with sea_response tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * rmeove unnecessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * improved models and filters from cloudfetch-sea branch Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * filters stuff (align with JDBC) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * backend from cloudfetch-sea Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove filtering, metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * raise NotImplementedErrror for metadata ops Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata commands Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * formatting (black) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add metadata command unit tests Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change to valid table name Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes covered by #588 Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * simplify test module Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * logging -> debug level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * change table name in log Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary backend cahnges Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-needed GetChunksResponse only relevant in Fetch phase Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication in response parsing Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * reduce code duplication Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * more clear docstrings Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * introduce strongly typed ChunkInfo Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove is_volume_operation from response Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add is_volume_op and more ResultData fields Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * add test scripts Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * Revert "Merge branch 'sea-migration' into exec-models-sea" This reverts commit 8bd12d829ea13abf8fc1507fff8cb21751001c67, reversing changes made to 030edf8df3db487b7af8d910ee51240d1339229e. * Revert "Merge branch 'exec-models-sea' into exec-phase-sea" This reverts commit be1997e0d6b6cf0f5499db2381971ec3a015a2f7, reversing changes made to 37813ba6d1fe06d7f9f10d510a059b88dc552496. * change logging level Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove un-necessary changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove excess changes Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * remove _get_schema_bytes (for now) Signed-off-by: varun-edachali-dbx <varun.edachali@databricks.com> * redundant comment…
1 parent c6f4a27 commit 141a004

32 files changed

+4098
-224
lines changed
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
"""
2+
Main script to run all SEA connector tests.
3+
4+
This script runs all the individual test modules and displays
5+
a summary of test results with visual indicators.
6+
7+
In order to run the script, the following environment variables need to be set:
8+
- DATABRICKS_SERVER_HOSTNAME: The hostname of the Databricks server
9+
- DATABRICKS_HTTP_PATH: The HTTP path of the Databricks server
10+
- DATABRICKS_TOKEN: The token to use for authentication
11+
"""
12+
13+
import os
14+
import sys
15+
import logging
16+
import subprocess
17+
from typing import List, Tuple
18+
19+
logging.basicConfig(level=logging.DEBUG)
20+
logger = logging.getLogger(__name__)
21+
22+
TEST_MODULES = [
23+
"test_sea_session",
24+
"test_sea_sync_query",
25+
"test_sea_async_query",
26+
"test_sea_metadata",
27+
]
28+
29+
30+
def run_test_module(module_name: str) -> bool:
31+
"""Run a test module and return success status."""
32+
module_path = os.path.join(
33+
os.path.dirname(os.path.abspath(__file__)), "tests", f"{module_name}.py"
34+
)
35+
36+
# Simply run the module as a script - each module handles its own test execution
37+
result = subprocess.run(
38+
[sys.executable, module_path], capture_output=True, text=True
39+
)
40+
41+
# Log the output from the test module
42+
if result.stdout:
43+
for line in result.stdout.strip().split("\n"):
44+
logger.info(line)
45+
46+
if result.stderr:
47+
for line in result.stderr.strip().split("\n"):
48+
logger.error(line)
49+
50+
return result.returncode == 0
51+
52+
53+
def run_tests() -> List[Tuple[str, bool]]:
54+
"""Run all tests and return results."""
55+
results = []
56+
57+
for module_name in TEST_MODULES:
58+
try:
59+
logger.info(f"\n{'=' * 50}")
60+
logger.info(f"Running test: {module_name}")
61+
logger.info(f"{'-' * 50}")
62+
63+
success = run_test_module(module_name)
64+
results.append((module_name, success))
65+
66+
status = "✅ PASSED" if success else "❌ FAILED"
67+
logger.info(f"Test {module_name}: {status}")
68+
69+
except Exception as e:
70+
logger.error(f"Error loading or running test {module_name}: {str(e)}")
71+
import traceback
72+
73+
logger.error(traceback.format_exc())
74+
results.append((module_name, False))
75+
76+
return results
77+
78+
79+
def print_summary(results: List[Tuple[str, bool]]) -> None:
80+
"""Print a summary of test results."""
81+
logger.info(f"\n{'=' * 50}")
82+
logger.info("TEST SUMMARY")
83+
logger.info(f"{'-' * 50}")
84+
85+
passed = sum(1 for _, success in results if success)
86+
total = len(results)
87+
88+
for module_name, success in results:
89+
status = "✅ PASSED" if success else "❌ FAILED"
90+
logger.info(f"{status} - {module_name}")
91+
92+
logger.info(f"{'-' * 50}")
93+
logger.info(f"Total: {total} | Passed: {passed} | Failed: {total - passed}")
94+
logger.info(f"{'=' * 50}")
95+
96+
97+
if __name__ == "__main__":
98+
# Check if required environment variables are set
99+
required_vars = [
100+
"DATABRICKS_SERVER_HOSTNAME",
101+
"DATABRICKS_HTTP_PATH",
102+
"DATABRICKS_TOKEN",
103+
]
104+
missing_vars = [var for var in required_vars if not os.environ.get(var)]
105+
106+
if missing_vars:
107+
logger.error(
108+
f"Missing required environment variables: {', '.join(missing_vars)}"
109+
)
110+
logger.error("Please set these variables before running the tests.")
111+
sys.exit(1)
112+
113+
# Run all tests
114+
results = run_tests()
115+
116+
# Print summary
117+
print_summary(results)
118+
119+
# Exit with appropriate status code
120+
all_passed = all(success for _, success in results)
121+
sys.exit(0 if all_passed else 1)

examples/experimental/tests/__init__.py

Whitespace-only changes.
Lines changed: 192 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,192 @@
1+
"""
2+
Test for SEA asynchronous query execution functionality.
3+
"""
4+
import os
5+
import sys
6+
import logging
7+
import time
8+
from databricks.sql.client import Connection
9+
from databricks.sql.backend.types import CommandState
10+
11+
logging.basicConfig(level=logging.INFO)
12+
logger = logging.getLogger(__name__)
13+
14+
15+
def test_sea_async_query_with_cloud_fetch():
16+
"""
17+
Test executing a query asynchronously using the SEA backend with cloud fetch enabled.
18+
19+
This function connects to a Databricks SQL endpoint using the SEA backend,
20+
executes a simple query asynchronously with cloud fetch enabled, and verifies that execution completes successfully.
21+
"""
22+
server_hostname = os.environ.get("DATABRICKS_SERVER_HOSTNAME")
23+
http_path = os.environ.get("DATABRICKS_HTTP_PATH")
24+
access_token = os.environ.get("DATABRICKS_TOKEN")
25+
catalog = os.environ.get("DATABRICKS_CATALOG")
26+
27+
if not all([server_hostname, http_path, access_token]):
28+
logger.error("Missing required environment variables.")
29+
logger.error(
30+
"Please set DATABRICKS_SERVER_HOSTNAME, DATABRICKS_HTTP_PATH, and DATABRICKS_TOKEN."
31+
)
32+
return False
33+
34+
try:
35+
# Create connection with cloud fetch enabled
36+
logger.info(
37+
"Creating connection for asynchronous query execution with cloud fetch enabled"
38+
)
39+
connection = Connection(
40+
server_hostname=server_hostname,
41+
http_path=http_path,
42+
access_token=access_token,
43+
catalog=catalog,
44+
schema="default",
45+
use_sea=True,
46+
user_agent_entry="SEA-Test-Client",
47+
use_cloud_fetch=True,
48+
enable_query_result_lz4_compression=False,
49+
)
50+
51+
logger.info(
52+
f"Successfully opened SEA session with ID: {connection.get_session_id_hex()}"
53+
)
54+
55+
# Execute a simple query asynchronously
56+
cursor = connection.cursor()
57+
logger.info(
58+
"Executing asynchronous query with cloud fetch: SELECT 1 as test_value"
59+
)
60+
cursor.execute_async("SELECT 1 as test_value")
61+
logger.info(
62+
"Asynchronous query submitted successfully with cloud fetch enabled"
63+
)
64+
65+
# Check query state
66+
logger.info("Checking query state...")
67+
while cursor.is_query_pending():
68+
logger.info("Query is still pending, waiting...")
69+
time.sleep(1)
70+
71+
logger.info("Query is no longer pending, getting results...")
72+
cursor.get_async_execution_result()
73+
logger.info(
74+
"Successfully retrieved asynchronous query results with cloud fetch enabled"
75+
)
76+
77+
# Close resources
78+
cursor.close()
79+
connection.close()
80+
logger.info("Successfully closed SEA session")
81+
82+
return True
83+
84+
except Exception as e:
85+
logger.error(
86+
f"Error during SEA asynchronous query execution test with cloud fetch: {str(e)}"
87+
)
88+
import traceback
89+
90+
logger.error(traceback.format_exc())
91+
return False
92+
93+
94+
def test_sea_async_query_without_cloud_fetch():
95+
"""
96+
Test executing a query asynchronously using the SEA backend with cloud fetch disabled.
97+
98+
This function connects to a Databricks SQL endpoint using the SEA backend,
99+
executes a simple query asynchronously with cloud fetch disabled, and verifies that execution completes successfully.
100+
"""
101+
server_hostname = os.environ.get("DATABRICKS_SERVER_HOSTNAME")
102+
http_path = os.environ.get("DATABRICKS_HTTP_PATH")
103+
access_token = os.environ.get("DATABRICKS_TOKEN")
104+
catalog = os.environ.get("DATABRICKS_CATALOG")
105+
106+
if not all([server_hostname, http_path, access_token]):
107+
logger.error("Missing required environment variables.")
108+
logger.error(
109+
"Please set DATABRICKS_SERVER_HOSTNAME, DATABRICKS_HTTP_PATH, and DATABRICKS_TOKEN."
110+
)
111+
return False
112+
113+
try:
114+
# Create connection with cloud fetch disabled
115+
logger.info(
116+
"Creating connection for asynchronous query execution with cloud fetch disabled"
117+
)
118+
connection = Connection(
119+
server_hostname=server_hostname,
120+
http_path=http_path,
121+
access_token=access_token,
122+
catalog=catalog,
123+
schema="default",
124+
use_sea=True,
125+
user_agent_entry="SEA-Test-Client",
126+
use_cloud_fetch=False,
127+
enable_query_result_lz4_compression=False,
128+
)
129+
130+
logger.info(
131+
f"Successfully opened SEA session with ID: {connection.get_session_id_hex()}"
132+
)
133+
134+
# Execute a simple query asynchronously
135+
cursor = connection.cursor()
136+
logger.info(
137+
"Executing asynchronous query without cloud fetch: SELECT 1 as test_value"
138+
)
139+
cursor.execute_async("SELECT 1 as test_value")
140+
logger.info(
141+
"Asynchronous query submitted successfully with cloud fetch disabled"
142+
)
143+
144+
# Check query state
145+
logger.info("Checking query state...")
146+
while cursor.is_query_pending():
147+
logger.info("Query is still pending, waiting...")
148+
time.sleep(1)
149+
150+
logger.info("Query is no longer pending, getting results...")
151+
cursor.get_async_execution_result()
152+
logger.info(
153+
"Successfully retrieved asynchronous query results with cloud fetch disabled"
154+
)
155+
156+
# Close resources
157+
cursor.close()
158+
connection.close()
159+
logger.info("Successfully closed SEA session")
160+
161+
return True
162+
163+
except Exception as e:
164+
logger.error(
165+
f"Error during SEA asynchronous query execution test without cloud fetch: {str(e)}"
166+
)
167+
import traceback
168+
169+
logger.error(traceback.format_exc())
170+
return False
171+
172+
173+
def test_sea_async_query_exec():
174+
"""
175+
Run both asynchronous query tests and return overall success.
176+
"""
177+
with_cloud_fetch_success = test_sea_async_query_with_cloud_fetch()
178+
logger.info(
179+
f"Asynchronous query with cloud fetch: {'✅ PASSED' if with_cloud_fetch_success else '❌ FAILED'}"
180+
)
181+
182+
without_cloud_fetch_success = test_sea_async_query_without_cloud_fetch()
183+
logger.info(
184+
f"Asynchronous query without cloud fetch: {'✅ PASSED' if without_cloud_fetch_success else '❌ FAILED'}"
185+
)
186+
187+
return with_cloud_fetch_success and without_cloud_fetch_success
188+
189+
190+
if __name__ == "__main__":
191+
success = test_sea_async_query_exec()
192+
sys.exit(0 if success else 1)

0 commit comments

Comments
 (0)