Skip to content

Commit def787e

Browse files
jgaucher-csgeonuxgithub-actions[bot]nleconte-csgroup
authored
fix issue + implement ci/cd and pytests + changes for rspy (#2)
* feat: remove health endpoint from swagger * fix: add aiohttp dependency * feat: add check_api_key endpiont * feat: add a rate limiter * feat: fix iam roles, change endpoints doc * feat: use oauth2 instead of openidconnect * feat: use implicit oauth2 workflow * fix: take Vincent remarks into account * feat: renew/revoke key: check the owner id first * feat: update database apikey from keycloak * fix: update database apikey from keycloak * update helm charts and dockerfile * fix: add log info * feat: /docs path for cluster deployment * feat: url prefix for cluster deployment * fix: add pool_pre_ping=True and email-validator * feat: add the user login to the api key information * feat: init the KeycloakOpenIDConnection on demand * add comments * workaround: disable refresh iam roles * fix suggestion for keycloak sync that reactivates key * feat: use authlib oauth authentication * fix: pre-commit * feat: redirect to /docs (without query params) after authent * feat: move openapi.json under /docs * feat: merge rspy branch into main * feat: customize the description in the swagger * implement pytests * implement pytests * feat: swagger description for the "config" field * implement pytests * implement pytests * Update latest_sync_date in renew_key * implement pytests * configure the rate limit * clean some code * Add the 'user_active" field = is user active in keycloak + add pytests * pytest: use cache in test * Automatic version with setuptools_scm * ci/cd: check code quality * fix ci/cd * ci/cd: publish docker images and helm charts * ci/cd: publish helm charts * Add APIKM_RATE_LIMIT env var to readme and helm chart * ci/cd: publish helm charts * update Helm documentation * ci/cd: publish helm charts * update Helm documentation * update Helm documentation * update Helm documentation * update Helm documentation * update Helm documentation * update Helm documentation * update Helm documentation * ci/cd: publish helm charts * update Helm documentation * update Helm documentation * Update helm chart values.yaml * update Helm documentation * update pytests for user_login * update Helm documentation * update Helm documentation * update Helm documentation * Delete commented code * update Helm documentation * add version number to fastapi * update Helm documentation * customize contact name, url and email * update Helm documentation * update helm charts for new env vars * update Helm documentation * update Helm documentation * update Helm documentation * pre-commit run --all-files * update Helm documentation * add comment * update Helm documentation * Update user infro from keycloak in renew_key * Externalize user_active * update Helm documentation * Fix version conflict (#7) * update Helm documentation * fix version conflict between setuptools_scm and helm * tests * generated * generated * test * test * Bot action: update version in helm chart. * update Helm documentation * Bot action: update version in helm chart. * update Helm documentation * test * Update version in helm chart. * update Helm documentation * Update version in helm chart. * test * test * test * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * test * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * test * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * Update version in helm chart. * update Helm documentation * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * test * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * configure git bot user * Update version in helm chart * update Helm documentation * Replace .dev1 by .dev0 * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * set docker image tag to latest * fix pytest (add user_login) * change workflow name * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * encrypt user_id and user_login in cookies * remove additional cryptography, keep logout from keycloak * chart.yaml appVersion = semver2 formating * Update version in helm chart * update Helm documentation * use random string to encode session cookies * Update version in helm chart * update Helm documentation * Update version in helm chart * update Helm documentation * improve security for default cookie secret * fix: helm templating Fix for the ingress' apiVersion in the copyright header * /auth/check_key is a technical endpoint * ignore safety issues * pre-commit --------- Co-authored-by: Vincent Gaudissart <vincent.gaudissart@csgroup.eu> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: nleconte-csgroup <86782407+nleconte-csgroup@users.noreply.github.com>
1 parent 2c4dfef commit def787e

31 files changed

+1922
-220
lines changed

‎.github/actions/pip-install/action.yml

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Copyright 2023-2024, CS GROUP - France, https://www.csgroup.eu/
2+
#
3+
# This file is part of APIKeyManager project
4+
# https://github.com/csgroup-oss/apikey-manager/
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
name: pip-install
19+
description: "'pip install' this project and dependencies"
20+
21+
inputs:
22+
options:
23+
description: "'pip install' <options> will run 'pip install -e .<options> --no-cache-dir'"
24+
required: false
25+
default: "[dev]"
26+
27+
working-directory:
28+
description: "the working directory in which poetry is run."
29+
required: false
30+
default: .
31+
32+
runs:
33+
using: "composite"
34+
steps:
35+
- uses: ./.github/actions/install-python
36+
- run: |
37+
pip install -e .${{ inputs.options }} --no-cache-dir
38+
working-directory: ${{ inputs.working-directory }}
39+
shell: bash

‎.github/actions/publish-chart/action.yml

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Copyright 2023-2024, CS GROUP - France, https://www.csgroup.eu/
2+
#
3+
# This file is part of APIKeyManager project
4+
# https://github.com/csgroup-oss/apikey-manager/
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
name: publish-chart
19+
description: Publish Helm chart
20+
21+
inputs:
22+
charts_dir:
23+
description: Directory containing charts to publish
24+
required: true
25+
github_token:
26+
description: secrets.GITHUB_TOKEN
27+
required: true
28+
29+
runs:
30+
using: "composite"
31+
steps:
32+
# See: https://github.com/helm/chart-releaser-action
33+
34+
- name: Configure Git
35+
run: |
36+
git config user.name "github-actions[bot]"
37+
git config user.email "github-actions[bot]@users.noreply.github.com"
38+
shell: bash
39+
40+
- name: Install Helm
41+
uses: azure/setup-helm@v4
42+
env:
43+
GITHUB_TOKEN: ${{ inputs.github_token }}
44+
45+
- name: Run chart-releaser
46+
uses: helm/chart-releaser-action@v1.6.0
47+
with:
48+
charts_dir: ${{ inputs.charts_dir }}
49+
env:
50+
CR_TOKEN: ${{ inputs.github_token }}
51+
CR_SKIP_EXISTING: false

‎.github/actions/publish-docker/action.yml

Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
# Copyright 2023-2024, CS GROUP - France, https://www.csgroup.eu/
2+
#
3+
# This file is part of APIKeyManager project
4+
# https://github.com/csgroup-oss/apikey-manager/
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
name: publish-docker
19+
description: Publish Docker image
20+
21+
inputs:
22+
dockerfile:
23+
description: Dockerfile path
24+
required: true
25+
build_context_path:
26+
description: "'docker build' context path"
27+
required: true
28+
build-args:
29+
description: List of build-time variables
30+
required: false
31+
image_suffix:
32+
description: Docker image name suffix
33+
required: false
34+
version_name:
35+
description: Version name e.g. 1.2.3a4.dev1a2b3c4d
36+
required: true
37+
github_token:
38+
description: secrets.GITHUB_TOKEN
39+
required: true
40+
other_docker_tag:
41+
description: Other tags for the docker images e.g. 'latest'
42+
required: false
43+
44+
outputs:
45+
docker_image:
46+
description: Docker image name:tag
47+
value: ${{ steps.docker_image.outputs.docker_image }}
48+
49+
runs:
50+
using: "composite"
51+
steps:
52+
# Replace invalid characters in the Docker version name, e.g. 1.2.3a4+dev1a2b3c4d becomes 1.2.3a4.dev1a2b3c4d
53+
# Then we can use ${{ inputs.docker_version_name }}
54+
- run: echo "docker_version_name=$(echo ${{ inputs.version_name }} | tr + .)" >> $GITHUB_ENV
55+
shell: bash
56+
57+
# Full Docker image name:tag as ghcr.io/csgroup-oss/apikey-manager<suffix>:<version> in lowercase
58+
- id: docker_image
59+
run: |
60+
docker_image=${{ env.DOCKER_REGISTRY }}/${{ github.repository }}${{ inputs.image_suffix }}:${{ env.docker_version_name }}
61+
docker_image=${docker_image,,} # lowercase
62+
echo docker_image=${docker_image} >> $GITHUB_ENV
63+
echo docker_image=${docker_image} >> $GITHUB_OUTPUT
64+
shell: bash
65+
66+
# Extract metadata from Git reference and GitHub events
67+
- name: Extract Docker metadata
68+
id: meta
69+
uses: docker/metadata-action@v5
70+
with:
71+
images: ${{ env.DOCKER_REGISTRY }}/${{ github.repository }} # ghcr.io/csgroup-oss/apikey-manager
72+
73+
# Checkout code from the current branch and build Docker image.
74+
- name: Build Docker image
75+
uses: docker/build-push-action@v5
76+
with:
77+
context: ${{ inputs.build_context_path }}
78+
file: ${{ inputs.dockerfile }}
79+
load: true
80+
build-args: ${{ inputs.build-args }}
81+
tags: ${{ env.docker_image }}
82+
labels: ${{ steps.meta.outputs.labels }}
83+
push: false # push after the security scans below
84+
85+
- name: Run Trivy vulnerability scanner
86+
uses: aquasecurity/trivy-action@master
87+
with:
88+
image-ref: ${{ env.docker_image }}
89+
format: sarif
90+
output: trivy-results-docker.sarif
91+
exit-code: 0
92+
#severity: HIGH,CRITICAL
93+
#timeout: '30m'
94+
95+
- name: Upload Trivy scan results to GitHub Security tab
96+
uses: github/codeql-action/upload-sarif@v3
97+
with:
98+
sarif_file: trivy-results-docker.sarif
99+
category: ${{ env.docker_image }}
100+
101+
- name: Display link to Trivy results
102+
run: |
103+
104+
set -x
105+
106+
# If this is not a pull request, the query is "is:open+branch:branch_name"
107+
if [[ "${{ github.ref_name }}" != *merge* ]]; then
108+
query="is:open+branch:${{ github.ref_name }}"
109+
110+
# Else the ref_name is e.g. '13/merge'. Change it into 'pr:13'
111+
else
112+
query=$(sed "s|\(.*\)/merge|pr:\1|g" <<< "${{ github.ref_name }}")
113+
fi
114+
115+
echo "Trivy scan results:" \
116+
"https://github.com/${{ github.repository }}/security/code-scanning?query=${query}" \
117+
>> $GITHUB_STEP_SUMMARY
118+
shell: bash
119+
120+
- name: Log into Docker registry
121+
uses: docker/login-action@v3
122+
with:
123+
registry: ${{ env.DOCKER_REGISTRY }}
124+
username: ${{ github.actor }}
125+
password: ${{ inputs.github_token }}
126+
127+
- name: Push Docker image
128+
run: docker push ${{ env.docker_image }}
129+
shell: bash
130+
131+
# Add a docker image tag and push it into the registry.
132+
- name: Push Docker tag e.g. 'latest'
133+
if: always()
134+
run: | # use sed to replace the last :<tag> by :other_docker_tag
135+
set -x
136+
if [[ ${{ inputs.other_docker_tag }}test != test ]]; then # if variable is defined and not empty
137+
docker_image_tag=$(sed "s|\(:[^:]*\)|:${{ inputs.other_docker_tag }}|g" <<< ${{ env.docker_image }})
138+
docker tag ${{ env.docker_image }} ${docker_image_tag}
139+
docker push ${docker_image_tag}
140+
fi
141+
shell: bash

‎.github/workflows/check-code-quality.yml

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# Copyright 2023-2024, CS GROUP - France, https://www.csgroup.eu/
2+
#
3+
# This file is part of APIKeyManager project
4+
# https://github.com/csgroup-oss/apikey-manager/
5+
#
6+
# Licensed under the Apache License, Version 2.0 (the "License");
7+
# you may not use this file except in compliance with the License.
8+
# You may obtain a copy of the License at
9+
#
10+
# http://www.apache.org/licenses/LICENSE-2.0
11+
#
12+
# Unless required by applicable law or agreed to in writing, software
13+
# distributed under the License is distributed on an "AS IS" BASIS,
14+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15+
# See the License for the specific language governing permissions and
16+
# limitations under the License.
17+
18+
name: Check code quality
19+
20+
on:
21+
push:
22+
pull_request:
23+
types: [opened, synchronize, reopened]
24+
workflow_dispatch:
25+
26+
env:
27+
PYTHON_VERSION: 3.11
28+
29+
jobs:
30+
check-format:
31+
runs-on: ubuntu-latest
32+
name: Check pre-commit
33+
steps:
34+
- uses: actions/checkout@v4
35+
- uses: ./.github/actions/install-python
36+
- run: >
37+
echo
38+
":information_source: This job checks that you have run \`pre-commit run --all-files\` in
39+
your local git repository before committing."
40+
>> $GITHUB_STEP_SUMMARY
41+
- uses: pre-commit/action@v3.0.1
42+
43+
check-license:
44+
runs-on: ubuntu-latest
45+
name: Check copyright license headers
46+
steps:
47+
- uses: actions/checkout@v4
48+
- run: |
49+
docker run -v ${{ github.workspace }}:/src ghcr.io/google/addlicense -check .
50+
51+
check-security:
52+
runs-on: ubuntu-latest
53+
name: Check security (bandit, safety, trivy)
54+
continue-on-error: true # run other jobs, resolve issues later
55+
permissions: write-all
56+
steps:
57+
- uses: actions/checkout@v4
58+
- uses: ./.github/actions/pip-install
59+
60+
- id: bandit
61+
name: Run bandit
62+
if: always() # even if previous steps returned a non-zero exit code
63+
run: |
64+
set -x
65+
python -m bandit -c pyproject.toml -r .
66+
shell: bash
67+
68+
- id: safety
69+
name: Run safety
70+
if: always()
71+
run: python -m safety check --full-report
72+
shell: bash
73+
74+
- name: Run Trivy vulnerability scanner
75+
if: always()
76+
uses: aquasecurity/trivy-action@master
77+
with:
78+
scan-type: fs
79+
ignore-unfixed: true
80+
format: sarif
81+
output: trivy-results-fs.sarif
82+
exit-code: 1
83+
#severity: 'CRITICAL'
84+
85+
- name: Upload Trivy scan results to GitHub Security tab
86+
if: always()
87+
uses: github/codeql-action/upload-sarif@v3
88+
with:
89+
sarif_file: trivy-results-fs.sarif
90+
category: git repository
91+
92+
- name: "Display link to Trivy results"
93+
if: always()
94+
run: |
95+
96+
set -x
97+
98+
# If this is not a pull request, the query is "is:open+branch:branch_name"
99+
if [[ "${{ github.ref_name }}" != *merge* ]]; then
100+
query="is:open+branch:${{ github.ref_name }}"
101+
102+
# Else the ref_name is e.g. '13/merge'. Change it into 'pr:13'
103+
else
104+
query=$(sed "s|\(.*\)/merge|pr:\1|g" <<< "${{ github.ref_name }}")
105+
fi
106+
107+
echo "Trivy scan results:" \
108+
"https://github.com/${{ github.repository }}/security/code-scanning?query=${query}" \
109+
>> $GITHUB_STEP_SUMMARY
110+
shell: bash
111+
112+
pytests:
113+
runs-on: ubuntu-latest
114+
name: Run pytest
115+
steps:
116+
- uses: actions/checkout@v4
117+
- uses: ./.github/actions/pip-install
118+
119+
- name: Run pytest
120+
run: |
121+
set -x && python -m pytest ./tests \
122+
--durations=0 \
123+
--error-for-skips \
124+
--cov=./app \
125+
--cov-report=term \
126+
--cov-report=xml:./cov-report.xml \
127+
shell: bash
128+
129+
- name: Display code coverage summary in this console
130+
uses: irongut/CodeCoverageSummary@v1.3.0
131+
with: # see https://github.com/marketplace/actions/code-coverage-summary#inputs
132+
filename: ./cov-report.xml

0 commit comments

Comments
 (0)