Skip to content

Commit ee7f1e3

Browse files
committed
Bumped up the version to 4.0.0.b3 and also changed the structure to have pyarrow as optional
1 parent 3d1ef79 commit ee7f1e3

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+2403
-1111
lines changed

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,20 @@
11
# Release History
22

3+
# 3.6.0 (2024-10-25)
4+
5+
- Support encryption headers in the cloud fetch request (https://github.com/databricks/databricks-sql-python/pull/460 by @jackyhu-db)
6+
7+
# 3.5.0 (2024-10-18)
8+
9+
- Create a non pyarrow flow to handle small results for the column set (databricks/databricks-sql-python#440 by @jprakash-db)
10+
- Fix: On non-retryable error, ensure PySQL includes useful information in error (databricks/databricks-sql-python#447 by @shivam2680)
11+
12+
# 3.4.0 (2024-08-27)
13+
14+
- Unpin pandas to support v2.2.2 (databricks/databricks-sql-python#416 by @kfollesdal)
15+
- Make OAuth as the default authenticator if no authentication setting is provided (databricks/databricks-sql-python#419 by @jackyhu-db)
16+
- Fix (regression): use SSL options with HTTPS connection pool (databricks/databricks-sql-python#425 by @kravets-levko)
17+
318
# 3.3.0 (2024-07-18)
419

520
- Don't retry requests that fail with HTTP code 401 (databricks/databricks-sql-python#408 by @Hodnebo)

CONTRIBUTING.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -85,18 +85,18 @@ We use [Pytest](https://docs.pytest.org/en/7.1.x/) as our test runner. Invoke it
8585
Unit tests do not require a Databricks account.
8686

8787
```bash
88-
poetry run python -m pytest databricks_sql_connector_core/tests/unit
88+
poetry run python -m pytest tests/unit
8989
```
9090
#### Only a specific test file
9191

9292
```bash
93-
poetry run python -m pytest databricks_sql_connector_core/tests/unit/tests.py
93+
poetry run python -m pytest tests/unit/tests.py
9494
```
9595

9696
#### Only a specific method
9797

9898
```bash
99-
poetry run python -m pytest databricks_sql_connector_core/tests/unit/tests.py::ClientTestSuite::test_closing_connection_closes_commands
99+
poetry run python -m pytest tests/unit/tests.py::ClientTestSuite::test_closing_connection_closes_commands
100100
```
101101

102102
#### e2e Tests
@@ -133,7 +133,7 @@ There are several e2e test suites available:
133133
To execute the core test suite:
134134

135135
```bash
136-
poetry run python -m pytest databricks_sql_connector_core/tests/e2e/driver_tests.py::PySQLCoreTestSuite
136+
poetry run python -m pytest tests/e2e/driver_tests.py::PySQLCoreTestSuite
137137
```
138138

139139
The `PySQLCoreTestSuite` namespace contains tests for all of the connector's basic features and behaviours. This is the default namespace where tests should be written unless they require specially configured clusters or take an especially long-time to execute by design.

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
[![PyPI](https://img.shields.io/pypi/v/databricks-sql-connector?style=flat-square)](https://pypi.org/project/databricks-sql-connector/)
44
[![Downloads](https://pepy.tech/badge/databricks-sql-connector)](https://pepy.tech/project/databricks-sql-connector)
55

6-
The Databricks SQL Connector for Python allows you to develop Python applications that connect to Databricks clusters and SQL warehouses. It is a Thrift-based client with no dependencies on ODBC or JDBC. It conforms to the [Python DB API 2.0 specification](https://www.python.org/dev/peps/pep-0249/) and exposes a [SQLAlchemy](https://www.sqlalchemy.org/) dialect for use with tools like `pandas` and `alembic` which use SQLAlchemy to execute DDL. Use `pip install databricks-sql-connector[databricks-sqlalchemy]` to install with SQLAlchemy's dependencies. `pip install databricks-sql-connector[alembic]` will install alembic's dependencies.
6+
The Databricks SQL Connector for Python allows you to develop Python applications that connect to Databricks clusters and SQL warehouses. It is a Thrift-based client with no dependencies on ODBC or JDBC. It conforms to the [Python DB API 2.0 specification](https://www.python.org/dev/peps/pep-0249/) and exposes a [SQLAlchemy](https://www.sqlalchemy.org/) dialect for use with tools like `pandas` and `alembic` which use SQLAlchemy to execute DDL. Use `pip install databricks-sql-connector[sqlalchemy]` to install with SQLAlchemy's dependencies. `pip install databricks-sql-connector[alembic]` will install alembic's dependencies.
77

88
This connector uses Arrow as the data-exchange format, and supports APIs to directly fetch Arrow tables. Arrow tables are wrapped in the `ArrowQueue` class to provide a natural API to get several rows at a time.
99

File renamed without changes.

databricks_sql_connector/pyproject.toml

Lines changed: 0 additions & 24 deletions
This file was deleted.

databricks_sql_connector_core/tests/unit/__init__.py

Whitespace-only changes.

examples/custom_cred_provider.py

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,27 @@
44
from databricks.sdk.oauth import OAuthClient
55
import os
66

7-
oauth_client = OAuthClient(host=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
8-
client_id=os.getenv("DATABRICKS_CLIENT_ID"),
9-
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"),
10-
redirect_url=os.getenv("APP_REDIRECT_URL"),
11-
scopes=['all-apis', 'offline_access'])
7+
oauth_client = OAuthClient(
8+
host=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
9+
client_id=os.getenv("DATABRICKS_CLIENT_ID"),
10+
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"),
11+
redirect_url=os.getenv("APP_REDIRECT_URL"),
12+
scopes=["all-apis", "offline_access"],
13+
)
1214

1315
consent = oauth_client.initiate_consent()
1416

1517
creds = consent.launch_external_browser()
1618

17-
with sql.connect(server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME"),
18-
http_path = os.getenv("DATABRICKS_HTTP_PATH"),
19-
credentials_provider=creds) as connection:
19+
with sql.connect(
20+
server_hostname=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
21+
http_path=os.getenv("DATABRICKS_HTTP_PATH"),
22+
credentials_provider=creds,
23+
) as connection:
2024

2125
for x in range(1, 5):
2226
cursor = connection.cursor()
23-
cursor.execute('SELECT 1+1')
27+
cursor.execute("SELECT 1+1")
2428
result = cursor.fetchall()
2529
for row in result:
2630
print(row)

examples/insert_data.py

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,23 @@
11
from databricks import sql
22
import os
33

4-
with sql.connect(server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME"),
5-
http_path = os.getenv("DATABRICKS_HTTP_PATH"),
6-
access_token = os.getenv("DATABRICKS_TOKEN")) as connection:
4+
with sql.connect(
5+
server_hostname=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
6+
http_path=os.getenv("DATABRICKS_HTTP_PATH"),
7+
access_token=os.getenv("DATABRICKS_TOKEN"),
8+
) as connection:
79

8-
with connection.cursor() as cursor:
9-
cursor.execute("CREATE TABLE IF NOT EXISTS squares (x int, x_squared int)")
10+
with connection.cursor() as cursor:
11+
cursor.execute("CREATE TABLE IF NOT EXISTS squares (x int, x_squared int)")
1012

11-
squares = [(i, i * i) for i in range(100)]
12-
values = ",".join([f"({x}, {y})" for (x, y) in squares])
13+
squares = [(i, i * i) for i in range(100)]
14+
values = ",".join([f"({x}, {y})" for (x, y) in squares])
1315

14-
cursor.execute(f"INSERT INTO squares VALUES {values}")
16+
cursor.execute(f"INSERT INTO squares VALUES {values}")
1517

16-
cursor.execute("SELECT * FROM squares LIMIT 10")
18+
cursor.execute("SELECT * FROM squares LIMIT 10")
1719

18-
result = cursor.fetchall()
20+
result = cursor.fetchall()
1921

20-
for row in result:
21-
print(row)
22+
for row in result:
23+
print(row)

examples/interactive_oauth.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,14 @@
1313
token across script executions.
1414
"""
1515

16-
with sql.connect(server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME"),
17-
http_path = os.getenv("DATABRICKS_HTTP_PATH")) as connection:
16+
with sql.connect(
17+
server_hostname=os.getenv("DATABRICKS_SERVER_HOSTNAME"),
18+
http_path=os.getenv("DATABRICKS_HTTP_PATH"),
19+
) as connection:
1820

1921
for x in range(1, 100):
2022
cursor = connection.cursor()
21-
cursor.execute('SELECT 1+1')
23+
cursor.execute("SELECT 1+1")
2224
result = cursor.fetchall()
2325
for row in result:
2426
print(row)

examples/m2m_oauth.py

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,17 +22,19 @@ def credential_provider():
2222
# Service Principal UUID
2323
client_id=os.getenv("DATABRICKS_CLIENT_ID"),
2424
# Service Principal Secret
25-
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"))
25+
client_secret=os.getenv("DATABRICKS_CLIENT_SECRET"),
26+
)
2627
return oauth_service_principal(config)
2728

2829

2930
with sql.connect(
30-
server_hostname=server_hostname,
31-
http_path=os.getenv("DATABRICKS_HTTP_PATH"),
32-
credentials_provider=credential_provider) as connection:
31+
server_hostname=server_hostname,
32+
http_path=os.getenv("DATABRICKS_HTTP_PATH"),
33+
credentials_provider=credential_provider,
34+
) as connection:
3335
for x in range(1, 100):
3436
cursor = connection.cursor()
35-
cursor.execute('SELECT 1+1')
37+
cursor.execute("SELECT 1+1")
3638
result = cursor.fetchall()
3739
for row in result:
3840
print(row)

0 commit comments

Comments
 (0)