-
Notifications
You must be signed in to change notification settings - Fork 5
Python workflows #35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Python workflows #35
Changes from 35 commits
Commits
Show all changes
38 commits
Select commit
Hold shift + click to select a range
9b84c17
feat: Add Python testing
GeekMasher 59ed01c
feat: Update testing and add scripts
GeekMasher a943337
feat: Add initial linting support
GeekMasher d598602
fix: Update matrix steps
GeekMasher 601181f
fix: Set Matrix issue
GeekMasher 344afd7
fix: Update set matrix script
GeekMasher 14651b5
feat: add needs to jobs
GeekMasher 19c37b7
fix: Update inputs
GeekMasher d194616
fix: Update typo
GeekMasher 1f49e68
fix: Update output
GeekMasher 5cfc574
feat: Update and in-line again
GeekMasher c674411
feat: Add general Python workflow
GeekMasher 74ae7c2
feat: Add Patch-Release-Me config
GeekMasher 9e2380d
feat: Add better logging
GeekMasher 161d338
feat: Add better testing support
GeekMasher 47dba7c
feat: Add Python Vendor reusable workflow
GeekMasher e94f1de
fix: GitHub Token and Env vars
GeekMasher ba4a936
fix: Custom Property
GeekMasher d03474e
fix: Add checkout
GeekMasher a2e3b9e
fix: Output of the custom property
GeekMasher 9375669
feat: Add Update PR support
GeekMasher 46c7100
feat: Add permissions and update PR creation
GeekMasher 7c3bece
fix: Permissions
GeekMasher f91b7ab
feat: Add change detection support
GeekMasher 32b441a
feat: Update Python workflow to support vendoring
GeekMasher 99ab119
feat: Python Release workflow
GeekMasher 5a87e27
feat: Add GitHub Releases
GeekMasher b61ce07
fix: Add GH Token
GeekMasher cbe2591
fix: Update version output
GeekMasher 6801d56
feat: Add GitHub Release
GeekMasher d1a5bc1
feat: Replace Python release with GitHub release
GeekMasher b4e19b3
fix: Add check
GeekMasher c7e3eb4
feat: Add release to Python workflow
GeekMasher bcae62d
feat: Remove release
GeekMasher 2c52d08
feat: Remove scripts
GeekMasher 027c609
Merge branch 'main' into python-workflows
GeekMasher b76c822
fix: Update typo
GeekMasher d9ccea8
fix: Update to main for now
GeekMasher File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
# Python Testing Workflow | ||
# | ||
# - Automatically runs tests on all supported versions of Python | ||
name: Python - Linting | ||
|
||
on: | ||
pull_request: | ||
workflow_call: | ||
inputs: | ||
tool: | ||
description: 'The tool to lint with' | ||
type: string | ||
default: 'ruff' | ||
versions: | ||
description: 'Python versions to test against' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.9,3.10,3.11,3.12' | ||
|
||
jobs: | ||
python-verions: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
steps: | ||
- name: Set matrix | ||
id: set-matrix | ||
run: | | ||
versions="${{ inputs.versions }}" | ||
echo "Version Input :: $versions" | ||
matrix=$(echo "$versions" | tr "," "\n" | awk '{print "\""$1"\""}' | paste -sd "," -) | ||
echo "matrix :: [$matrix]" | ||
echo "matrix=[$matrix]" >> "$GITHUB_OUTPUT" | ||
|
||
python-linting: | ||
runs-on: ubuntu-latest | ||
if: ${{ needs.python-verions.outputs.matrix != '[]' }} | ||
needs: [ python-verions ] | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ${{ fromJson(needs.python-verions.outputs.matrix) }} | ||
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
|
||
- name: Install dependencies | ||
run: | | ||
set -e | ||
if [[ -f pyproject.toml ]]; then | ||
python -m pip install --upgrade pip poetry | ||
poetry install | ||
elif [[ -f Pipfile ]]; then | ||
python -m pip install --upgrade pip pipenv | ||
pipenv sync -d | ||
elif [[ -f requirements.txt ]]; then | ||
python -m pip install --upgrade pip | ||
pip install -r requirements.txt | ||
elif [[ -f Makefile ]]; then | ||
make install | ||
else | ||
echo "No manifest files found to install dependencies" | ||
fi | ||
|
||
- name: Run linting | ||
run: | | ||
set -e | ||
TOOL="${{ inputs.tool }}" | ||
if [[ "$TOOL" == "ruff" ]]; then | ||
pip install ruff | ||
ruff check | ||
elif [[ "$TOOL" == "flake8" ]]; then | ||
pip install flake8 | ||
flake8 . | ||
elif [[ "$TOOL" == "black" ]]; then | ||
pip install black | ||
black --check . | ||
fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,65 @@ | ||
# Python Releasing Workflow | ||
name: Python - Release | ||
|
||
on: | ||
push: | ||
workflow_call: | ||
inputs: | ||
version: | ||
description: 'Python main version to vendor' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.11' | ||
|
||
permissions: | ||
contents: write | ||
pull-requests: write | ||
|
||
jobs: | ||
version-changes: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
release: ${{ steps.check_release.outputs.release }} | ||
version: ${{ steps.check_release.outputs.version }} | ||
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: "Check release" | ||
id: check_release | ||
env: | ||
GH_TOKEN: ${{ github.token }} | ||
run: | | ||
set -e | ||
|
||
if [[ -f .release.yml ]]; then | ||
pip install yq | ||
current_version=$(cat .release.yml | yq -r ".version") | ||
elif [[ -f pyproject.toml ]]; then | ||
current_version=$(grep -oP '^version = "(.*)"$' pyproject.toml | cut -d '"' -f 2) | ||
elif [[ -f setup.py ]]; then | ||
current_version=$(grep -oP '^__version__ = "(.*)"$' setup.py | cut -d '"' -f 2) | ||
else | ||
echo "No version file found" | ||
current_version="NA" | ||
fi | ||
|
||
released_version=$(gh api -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/:owner/:repo/releases/latest | jq -r ".tag_name") | ||
|
||
if [[ "$current_version" == "NA" || "$current_version" == "$released_version" ]]; then | ||
echo "No new release found" | ||
echo "release=false" >> "$GITHUB_OUTPUT" | ||
else | ||
echo "New release found" | ||
echo "version=$current_version" >> "$GITHUB_OUTPUT" | ||
echo "release=true" >> "$GITHUB_OUTPUT" | ||
fi | ||
|
||
github-release: | ||
uses: advanced-security/reusable-workflows/.github/workflows/release.yml@v0.1.0 | ||
GeekMasher marked this conversation as resolved.
Show resolved
Hide resolved
|
||
needs: [ version-changes ] | ||
if: ${{ needs.version-changes.outputs.release == 'true' }} | ||
secrets: inherit | ||
with: | ||
version: ${{ needs.version-changes.outputs.version }} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,85 @@ | ||
# Python Testing Workflow | ||
# | ||
# - Automatically runs tests on all supported versions of Python | ||
name: Python - Testing | ||
|
||
on: | ||
pull_request: | ||
workflow_call: | ||
inputs: | ||
versions: | ||
description: 'Python versions to test against' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.9,3.10,3.11,3.12' | ||
|
||
jobs: | ||
python-verions: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
matrix: ${{ steps.set-matrix.outputs.matrix }} | ||
steps: | ||
- name: Set matrix | ||
id: set-matrix | ||
run: | | ||
versions="${{ inputs.versions }}" | ||
echo "Version Input :: $versions" | ||
matrix=$(echo "$versions" | tr "," "\n" | awk '{print "\""$1"\""}' | paste -sd "," -) | ||
echo "matrix :: [$matrix]" | ||
echo "matrix=[$matrix]" >> "$GITHUB_OUTPUT" | ||
|
||
python-testing: | ||
# This workflow runs on the latest version of Ubuntu | ||
runs-on: ubuntu-latest | ||
if: ${{ needs.python-verions.outputs.matrix != '[]' }} | ||
needs: [ python-verions ] | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
python-version: ${{ fromJSON(needs.python-verions.outputs.matrix) }} | ||
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Set up Python ${{ matrix.python-version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ matrix.python-version }} | ||
|
||
- name: Install dependencies | ||
run: | | ||
set -e | ||
echo "Installing dependencies..." | ||
if [[ -f pyproject.toml ]]; then | ||
python -m pip install --upgrade pip poetry | ||
poetry install | ||
elif [[ -f Pipfile ]]; then | ||
python -m pip install --upgrade pip pipenv | ||
pipenv sync -d | ||
elif [[ -f requirements.txt ]]; then | ||
python -m pip install --upgrade pip | ||
pip install -r requirements.txt | ||
elif [[ -f Makefile ]]; then | ||
make install | ||
else | ||
echo "No manifest files found to install dependencies" | ||
fi | ||
|
||
- name: Run tests | ||
run: | | ||
set -e | ||
echo "Running Python tests..." | ||
if [[ -f pyproject.toml ]]; then | ||
echo "Running poetry run test" | ||
poetry run test | ||
elif [[ -f Pipfile ]]; then | ||
echo "Running pipenv run test" | ||
pipenv run test | ||
elif [[ -f Makefile ]]; then | ||
echo "Running make test" | ||
make test | ||
else | ||
echo "Unknown test runner..." | ||
echo "Please contact the oss-maintainers team for help." | ||
fi |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
# Python Vendoring Workflow | ||
name: Python - Vendoring | ||
|
||
on: | ||
push: | ||
pull_request: | ||
workflow_call: | ||
inputs: | ||
custom-property: | ||
description: 'Name of the custom property to get value from' | ||
type: string | ||
default: 'OSSType' | ||
version: | ||
description: 'Python main version to vendor' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.11' | ||
|
||
permissions: | ||
contents: write | ||
pull-requests: write | ||
|
||
jobs: | ||
custom-property: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
osstype: ${{ steps.get_custom_property.outputs.osstype }} | ||
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
- name: "Get Custom Property" | ||
id: get_custom_property | ||
env: | ||
GH_TOKEN: ${{ github.token }} | ||
PROPERTY_NAME: ${{ inputs.custom-property }} | ||
run: | | ||
set -e | ||
|
||
PROPERTIES=$(gh api -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/:owner/:repo/properties/values) | ||
echo "Properties: '$PROPERTIES'" | ||
|
||
REPOSITORY_TYPE=$(echo $PROPERTIES | jq -r ".[] | select(.property_name == \"$PROPERTY_NAME\") | .value") | ||
echo "Repository type: '$REPOSITORY_TYPE'" | ||
echo "osstype=$REPOSITORY_TYPE" >> "$GITHUB_OUTPUT" | ||
|
||
python-vendoring: | ||
runs-on: ubuntu-latest | ||
needs: [ custom-property ] | ||
if: ${{ needs.custom-property.outputs.osstype == 'Actions' }} | ||
GeekMasher marked this conversation as resolved.
Show resolved
Hide resolved
|
||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v4 | ||
|
||
- name: Set up Python ${{ inputs.version }} | ||
uses: actions/setup-python@v5 | ||
with: | ||
python-version: ${{ inputs.version }} | ||
|
||
- name: "Install and Vendor dependencies" | ||
id: vendoring | ||
run: | | ||
set -e | ||
|
||
if [[ -f Pipfile ]]; then | ||
python -m pip install --upgrade pip pipenv | ||
pipenv run vendor | ||
elif [[ -f Makefile ]]; then | ||
make vendor | ||
elif [[ -f vendor/update.sh ]]; then | ||
./vendor/update.sh | ||
else | ||
echo "Unknown vendoring method" | ||
fi | ||
|
||
CHANGES=$(git status --porcelain | wc -l) | ||
echo "changes=$CHANGES" >> "$GITHUB_OUTPUT" | ||
|
||
- name: "Update vendored dependencies (Push)" | ||
if: ${{ steps.vendoring.outputs.changes != 0 && (github.event_name == 'push' || github.event_name == 'workflow_dispatch') }} | ||
uses: peter-evans/create-pull-request@v6 | ||
with: | ||
token: ${{ github.token }} | ||
commit-message: "[chore]: Update vendored dependencies" | ||
title: "[chore]: Update vendored dependencies" | ||
branch: update-vendored-dependencies | ||
base: ${{ github.event.before }} | ||
labels: dependencies | ||
body: | | ||
This is an automated PR to update that vendored dependencies are up to date. | ||
It was created by a GitHub workflow defined in `.github/workflows/python-vendor.yml`. | ||
Please do not merge this PR manually. | ||
<details> | ||
<summary>Details</summary> | ||
<p> | ||
This PR was created by a workflow that runs on all pushes to the repository. | ||
It installs dependencies and then verifies that the repository is clean. | ||
</p> | ||
</details> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# Python Testing Workflow | ||
# | ||
# - Automatically runs tests on all supported versions of Python | ||
name: Python | ||
|
||
on: | ||
pull_request: | ||
workflow_call: | ||
inputs: | ||
version: | ||
description: 'Python main version to vendor' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.11' | ||
versions: | ||
description: 'Python versions to test against' | ||
type: string | ||
# All Major versions of Python that are currently supported | ||
default: '3.9,3.10,3.11,3.12' | ||
vendor: | ||
description: 'Whether to vendor the dependencies' | ||
type: string | ||
default: 'true' | ||
|
||
|
||
jobs: | ||
# Run the tests on all supported versions of Python | ||
testing: | ||
uses: advanced-security/reusable-workflows/.github/workflows/python-testing.yml@v0.1.0 | ||
secrets: inherit | ||
with: | ||
versions: ${{ inputs.versions }} | ||
|
||
# Run linters on the codebase | ||
linting: | ||
uses: advanced-security/reusable-workflows/.github/workflows/python-linting.yml@v0.1.0 | ||
needs: [ testing ] | ||
secrets: inherit | ||
with: | ||
versions: ${{ inputs.versions }} | ||
|
||
# Vendor the dependencies into the repository if needed | ||
vendoring: | ||
uses: advanced-security/reusable-workflows/.github/workflows/python-vendor.yml@v0.1.0 | ||
needs: [ testing, linting ] | ||
if: ${{ inputs.vendor == 'true' }} | ||
secrets: inherit | ||
with: | ||
version: ${{ inputs.version }} | ||
custom-property: 'OSSType' | ||
|
||
# Release a new version of the package | ||
release: | ||
uses: advanced-security/reusable-workflows/.github/workflows/python-release.yml@v0.1.0 | ||
needs: [ testing, linting ] | ||
secrets: inherit | ||
with: | ||
version: ${{ inputs.version }} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.