Skip to content

Commit 7cac9dc

Browse files
jprakash-dbjprakash-db
authored
Setting up Github Actions for Sqlalchemy v2 (#9)
* Added github actions * Moved tests to a separate folder * Added __init__.py in sqlalchemy folder * Ignore missing imports in mypy * Check * Check * Fixed readme.md * Minor fix
1 parent fef6428 commit 7cac9dc

26 files changed

+401
-3
lines changed

‎.github/.github/pull_request_template.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
<!-- We welcome contributions. All patches must include a sign-off. Please see CONTRIBUTING.md for details -->
2+
3+
4+
## What type of PR is this?
5+
<!-- Check all that apply, delete what doesn't apply. -->
6+
7+
- [ ] Refactor
8+
- [ ] Feature
9+
- [ ] Bug Fix
10+
- [ ] Other
11+
12+
## Description
13+
14+
## How is this tested?
15+
16+
- [ ] Unit tests
17+
- [ ] E2E Tests
18+
- [ ] Manually
19+
- [ ] N/A
20+
21+
<!-- If Manually, please describe. -->
22+
23+
## Related Tickets & Documents

‎.github/workflows/code-quality-checks.yml

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
name: Code Quality Checks
2+
on:
3+
push:
4+
branches:
5+
- main
6+
pull_request:
7+
branches:
8+
- main
9+
workflow_dispatch:
10+
jobs:
11+
check-linting:
12+
runs-on: ubuntu-latest
13+
strategy:
14+
matrix:
15+
python-version: [3.8, 3.9, "3.10"]
16+
steps:
17+
#----------------------------------------------
18+
# check-out repo and set-up python
19+
#----------------------------------------------
20+
- name: Check out repository
21+
uses: actions/checkout@v2
22+
- name: Set up python ${{ matrix.python-version }}
23+
id: setup-python
24+
uses: actions/setup-python@v2
25+
with:
26+
python-version: ${{ matrix.python-version }}
27+
#----------------------------------------------
28+
# ----- install & configure poetry -----
29+
#----------------------------------------------
30+
- name: Install Poetry
31+
uses: snok/install-poetry@v1
32+
with:
33+
virtualenvs-create: true
34+
virtualenvs-in-project: true
35+
installer-parallel: true
36+
37+
#----------------------------------------------
38+
# load cached venv if cache exists
39+
#----------------------------------------------
40+
- name: Load cached venv
41+
id: cached-poetry-dependencies
42+
uses: actions/cache@v2
43+
with:
44+
path: .venv
45+
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
46+
#----------------------------------------------
47+
# install dependencies if cache does not exist
48+
#----------------------------------------------
49+
- name: Install dependencies
50+
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
51+
run: poetry install --no-interaction --no-root
52+
#----------------------------------------------
53+
# install your root project, if required
54+
#----------------------------------------------
55+
- name: Install library
56+
run: poetry install --no-interaction
57+
#----------------------------------------------
58+
# black the code
59+
#----------------------------------------------
60+
- name: Black
61+
run: poetry run black --check src
62+
63+
check-types:
64+
runs-on: ubuntu-latest
65+
strategy:
66+
matrix:
67+
python-version: [3.8, 3.9, "3.10"]
68+
steps:
69+
#----------------------------------------------
70+
# check-out repo and set-up python
71+
#----------------------------------------------
72+
- name: Check out repository
73+
uses: actions/checkout@v2
74+
- name: Set up python ${{ matrix.python-version }}
75+
id: setup-python
76+
uses: actions/setup-python@v2
77+
with:
78+
python-version: ${{ matrix.python-version }}
79+
#----------------------------------------------
80+
# ----- install & configure poetry -----
81+
#----------------------------------------------
82+
- name: Install Poetry
83+
uses: snok/install-poetry@v1
84+
with:
85+
virtualenvs-create: true
86+
virtualenvs-in-project: true
87+
installer-parallel: true
88+
89+
#----------------------------------------------
90+
# load cached venv if cache exists
91+
#----------------------------------------------
92+
- name: Load cached venv
93+
id: cached-poetry-dependencies
94+
uses: actions/cache@v2
95+
with:
96+
path: .venv
97+
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
98+
#----------------------------------------------
99+
# install dependencies if cache does not exist
100+
#----------------------------------------------
101+
- name: Install dependencies
102+
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
103+
run: poetry install --no-interaction --no-root
104+
#----------------------------------------------
105+
# install your root project, if required
106+
#----------------------------------------------
107+
- name: Install library
108+
run: poetry install --no-interaction
109+
#----------------------------------------------
110+
# mypy the code
111+
#----------------------------------------------
112+
- name: Mypy
113+
run: |
114+
mkdir .mypy_cache # Workaround for bad error message "error: --install-types failed (no mypy cache directory)"; see https://github.com/python/mypy/issues/10768#issuecomment-2178450153
115+
poetry run mypy --install-types --non-interactive src

‎.github/workflows/dco-check.yml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
name: DCO Check
2+
3+
on: [pull_request]
4+
5+
jobs:
6+
check:
7+
runs-on: ubuntu-latest
8+
steps:
9+
- name: Check for DCO
10+
id: dco-check
11+
uses: tisonkun/actions-dco@v1.1
12+
- name: Comment about DCO status
13+
uses: actions/github-script@v6
14+
if: ${{ failure() }}
15+
with:
16+
script: |
17+
github.rest.issues.createComment({
18+
issue_number: context.issue.number,
19+
owner: context.repo.owner,
20+
repo: context.repo.repo,
21+
body: `Thanks for your contribution! To satisfy the DCO policy in our \
22+
[contributing guide](https://github.com/databricks/databricks-sqlalchemy/blob/main/CONTRIBUTING.md) \
23+
every commit message must include a sign-off message. One or more of your commits is missing this message. \
24+
You can reword previous commit messages with an interactive rebase (\`git rebase -i main\`).`
25+
})

‎.github/workflows/integration.yml

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
name: Integration Tests
2+
on:
3+
push:
4+
paths-ignore:
5+
- "**.MD"
6+
- "**.md"
7+
8+
jobs:
9+
run-e2e-tests:
10+
runs-on: ubuntu-latest
11+
environment: azure-prod
12+
env:
13+
DATABRICKS_SERVER_HOSTNAME: ${{ secrets.DATABRICKS_HOST }}
14+
DATABRICKS_HTTP_PATH: ${{ secrets.TEST_PECO_WAREHOUSE_HTTP_PATH }}
15+
DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }}
16+
DATABRICKS_CATALOG: peco
17+
DATABRICKS_USER: ${{ secrets.TEST_PECO_SP_ID }}
18+
steps:
19+
#----------------------------------------------
20+
# check-out repo and set-up python
21+
#----------------------------------------------
22+
- name: Check out repository
23+
uses: actions/checkout@v3
24+
- name: Set up python
25+
id: setup-python
26+
uses: actions/setup-python@v4
27+
with:
28+
python-version: "3.10"
29+
#----------------------------------------------
30+
# ----- install & configure poetry -----
31+
#----------------------------------------------
32+
- name: Install Poetry
33+
uses: snok/install-poetry@v1
34+
with:
35+
virtualenvs-create: true
36+
virtualenvs-in-project: true
37+
installer-parallel: true
38+
39+
#----------------------------------------------
40+
# load cached venv if cache exists
41+
#----------------------------------------------
42+
- name: Load cached venv
43+
id: cached-poetry-dependencies
44+
uses: actions/cache@v2
45+
with:
46+
path: .venv
47+
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
48+
#----------------------------------------------
49+
# install dependencies if cache does not exist
50+
#----------------------------------------------
51+
- name: Install dependencies
52+
run: poetry install --no-interaction --all-extras
53+
#----------------------------------------------
54+
# run test suite
55+
#----------------------------------------------
56+
- name: Run SQL Alchemy tests
57+
run: poetry run python -m pytest tests/test_local

‎.github/workflows/publish-test.yml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
name: Publish to PyPI [Test]
2+
on: [push]
3+
jobs:
4+
test-pypi:
5+
name: Create patch version number and push to test-pypi
6+
runs-on: ubuntu-latest
7+
steps:
8+
#----------------------------------------------
9+
# check-out repo and set-up python
10+
#----------------------------------------------
11+
- name: Check out repository
12+
uses: actions/checkout@v2
13+
- name: Set up python
14+
id: setup-python
15+
uses: actions/setup-python@v2
16+
with:
17+
python-version: 3.9
18+
#----------------------------------------------
19+
# ----- install & configure poetry -----
20+
#----------------------------------------------
21+
- name: Install Poetry
22+
uses: snok/install-poetry@v1
23+
with:
24+
virtualenvs-create: true
25+
virtualenvs-in-project: true
26+
installer-parallel: true
27+
#----------------------------------------------
28+
# load cached venv if cache exists
29+
#----------------------------------------------
30+
- name: Load cached venv
31+
id: cached-poetry-dependencies
32+
uses: actions/cache@v2
33+
with:
34+
path: .venv
35+
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
36+
#----------------------------------------------
37+
# install dependencies if cache does not exist
38+
#----------------------------------------------
39+
- name: Install dependencies
40+
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
41+
run: poetry install --no-interaction --no-root
42+
#----------------------------------------------
43+
# Get the current version and increment it (test-pypi requires a unique version number)
44+
#----------------------------------------------
45+
- name: Get next version
46+
uses: reecetech/version-increment@2022.2.4
47+
id: version
48+
with:
49+
scheme: semver
50+
increment: patch
51+
#----------------------------------------------
52+
# Tell poetry to update the version number
53+
#----------------------------------------------
54+
- name: Update pyproject.toml
55+
run: poetry version ${{ steps.version.outputs.major-version }}.${{ steps.version.outputs.minor-version }}.dev$(date +%s)
56+
57+
- name: Build and publish to pypi
58+
uses: JRubics/poetry-publish@v1.10
59+
with:
60+
pypi_token: ${{ secrets.SQLALCHEMY_TEST_PYPI_TOKEN }}
61+
repository_name: "testpypi"
62+
repository_url: "https://test.pypi.org/legacy/"

‎.github/workflows/publish.yml

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: Publish to PyPI [Production]
2+
on:
3+
release:
4+
types: [published]
5+
jobs:
6+
publish:
7+
name: Publish
8+
runs-on: ubuntu-latest
9+
if: github.ref == 'refs/heads/main'
10+
steps:
11+
#----------------------------------------------
12+
# check-out repo and set-up python
13+
#----------------------------------------------
14+
- name: Check out repository
15+
uses: actions/checkout@v2
16+
- name: Set up python
17+
id: setup-python
18+
uses: actions/setup-python@v2
19+
with:
20+
python-version: 3.9
21+
#----------------------------------------------
22+
# ----- install & configure poetry -----
23+
#----------------------------------------------
24+
- name: Install Poetry
25+
uses: snok/install-poetry@v1
26+
with:
27+
virtualenvs-create: true
28+
virtualenvs-in-project: true
29+
installer-parallel: true
30+
#----------------------------------------------
31+
# load cached venv if cache exists
32+
#----------------------------------------------
33+
- name: Load cached venv
34+
id: cached-poetry-dependencies
35+
uses: actions/cache@v2
36+
with:
37+
path: .venv
38+
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
39+
#----------------------------------------------
40+
# install dependencies if cache does not exist
41+
#----------------------------------------------
42+
- name: Install dependencies
43+
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
44+
run: poetry install --no-interaction --no-root
45+
#------------------------------------------------------------------------------------------------
46+
# Here we use version-increment to fetch the latest tagged version (we won't increment it though)
47+
#------------------------------------------------------------------------------------------------
48+
- name: Get next version
49+
uses: reecetech/version-increment@2022.2.4
50+
id: version
51+
with:
52+
scheme: semver
53+
increment: patch
54+
#-----------------------------------------------------------------------------
55+
# Tell poetry to use the `current-version` that was found by the previous step
56+
#-----------------------------------------------------------------------------
57+
- name: Update pyproject.toml
58+
run: poetry version ${{ steps.version.outputs.current-version }}
59+
#----------------------------------------------
60+
# Attempt push to test-pypi
61+
#----------------------------------------------
62+
- name: Build and publish to pypi
63+
uses: JRubics/poetry-publish@v1.10
64+
with:
65+
pypi_token: ${{ secrets.SQLALCHEMY_PROD_PYPI_TOKEN }}

‎README.tests.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ We maintain `_extra.py` with test cases that depend on SQLAlchemy's reusable dia
2323

2424
```
2525
poetry shell
26-
cd src/databricks/sqlalchemy/test
26+
cd tests/test
2727
python -m pytest test_suite.py --dburi \
2828
"databricks://token:$access_token@$host?http_path=$http_path&catalog=$catalog&schema=$schema"
2929
```

0 commit comments

Comments
 (0)