Skip to content

[PFMENG-2716] fix v2 for dualstack #106

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 6, 2025
Merged

[PFMENG-2716] fix v2 for dualstack #106

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
295 changes: 292 additions & 3 deletions .github/workflows/terraform.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@

jobs:
fmt-validate:
if: github.ref_name != 'main' && inputs.enable_format == true
if: github.ref_name != 'main' && inputs.enable_format == true && inputs.runner_label != 'platform-eng-ent-v2-dual'
name: Format and Validate
runs-on:
- ${{ inputs.default_runner_override_label }}
Expand Down Expand Up @@ -212,7 +212,7 @@

lint:
name: Linting
if: github.ref_name != 'main' && inputs.enable_lint == true
if: github.ref_name != 'main' && inputs.enable_lint == true && inputs.runner_label != 'platform-eng-ent-v2-dual'
runs-on:
- ${{ inputs.default_runner_override_label }}
- ${{ inputs.runner_label }}
Expand Down Expand Up @@ -340,7 +340,7 @@
#steps.tflint.outcome check for outcome
security:
name: Security Checks
if: github.ref_name != 'main' && inputs.enable_security_check == true
if: github.ref_name != 'main' && inputs.enable_security_check == true && inputs.runner_label != 'platform-eng-ent-v2-dual'
runs-on:
- ${{ inputs.default_runner_override_label }}
- ${{ inputs.runner_label }}
Expand Down Expand Up @@ -402,3 +402,292 @@
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'results.sarif'

fmt-validate_on_dualstack:
if: github.ref_name != 'main' && inputs.enable_format == true && inputs.runner_label == 'platform-eng-ent-v2-dual'
name: Format and Validate
runs-on:
- ${{ inputs.runner_label }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: ${{ inputs.enable_submodules }}

- uses: actions/setup-python@v5
with:
python-version: '3.11'

- run: mkdir -p "${TF_PLUGIN_CACHE_DIR}"
- name: Cache Terraform
uses: actions/cache@v4
with:
path: ${{ env.TF_PLUGIN_CACHE_DIR }}
key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}
- name: Cache TFLint plugin dir
uses: actions/cache@v4
with:
path: ~/.tflint.d/plugins
key: ${{ runner.os }}-tflint-${{ hashFiles('**/.tflint.hcl') }}

- name: Setup Node only for self-hosted runners
uses: actions/setup-node@v4
if: ${{ inputs.runner_label == 'platform-eng-ent-v2-dual' }}
with:
node-version: 18

- name: Setup Terraform
uses: hashicorp/setup-terraform@v3

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'hashicorp/setup-terraform' with ref 'v3', not a pinned commit hash
with:
terraform_version: latest
cli_config_credentials_token: ${{ secrets.TFE_TOKEN }}

- name: Setup Helm
uses: azure/setup-helm@v4

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'azure/setup-helm' with ref 'v4', not a pinned commit hash
with:
version: 'latest'
token: ${{ secrets.GITHUB_TOKEN }}

- name: Pre-init Hook
run: ${{ inputs.pre_init_hook }}

- name: Precommit Skips
id: precommit_skips
run: |
SKIPS="shellcheck,tflint,terraform_tflint,markdown-link-check,terraform_docs,terraform_tfsec,checkov,terraform_checkov"
if [ "${branch}" == "${main_branch}" ];then
SKIPS="${SKIPS},no-commit-to-branch"
fi
if [ "${{inputs.skip_precommit}}" != "" ]; then
SKIPS="${SKIPS},${{inputs.skip_precommit}}"
fi

echo "${SKIPS}"
echo "skips=${SKIPS}" >> "${GITHUB_OUTPUT}"
env:
branch: ${{ github.ref_name }}
main_branch: ${{ inputs.main_branch }}

- name: precommit run hooks
id: precommit_run_hooks #Run on all PRs from the same repo, fork repos to run all
if: inputs.pre_commit_run_all == false
env:
SKIP: ${{ steps.precommit_skips.outputs.skips }}
run: |
pip install pre-commit
git fetch origin
if [ "$GITHUB_EVENT_NAME" == 'pull_request' ]
then
if [ ${{ github.event.pull_request.head.repo.full_name }} == ${{ github.repository }} ]
then
pre-commit run --from-ref origin/${{ github.base_ref }} --to-ref ${{ github.event.pull_request.head.sha }} --color=always --show-diff-on-failure
else
pre-commit run --color=always --show-diff-on-failure --all-files
fi
else
pre-commit run --from-ref origin/${{ github.event.repository.default_branch }} --to-ref "$GITHUB_SHA" --color=always --show-diff-on-failure
fi

- name: precommit run tflint hooks
id: precommit_run_hooks_all
if: inputs.pre_commit_run_all
uses: pre-commit/action@v3.0.1

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step: precommit_run_hooks_all
Loading
uses 'pre-commit/action' with ref 'v3.0.1', not a pinned commit hash
env:
SKIP: ${{ steps.precommit_skips.outputs.skips }}
with:
extra_args: --color=always --show-diff-on-failure --all-files

- run: terraform -v

lint_on_dualstack:
name: Linting
if: github.ref_name != 'main' && inputs.enable_lint == true && inputs.runner_label == 'platform-eng-ent-v2-dual'
runs-on:
- ${{ inputs.runner_label }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: ${{ inputs.enable_submodules }}

- uses: actions/setup-python@v5
with:
python-version: '3.11'

- run: mkdir -p "${TF_PLUGIN_CACHE_DIR}"
- name: Cache Terraform
uses: actions/cache@v4
with:
path: ${{ env.TF_PLUGIN_CACHE_DIR }}
key: ${{ runner.os }}-terraform-${{ hashFiles('**/.terraform.lock.hcl') }}

- name: Setup Terraform
uses: hashicorp/setup-terraform@v3

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'hashicorp/setup-terraform' with ref 'v3', not a pinned commit hash
with:
terraform_version: latest
cli_config_credentials_token: ${{ secrets.TFE_TOKEN }}

- name: Setup Helm
uses: azure/setup-helm@v4

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'azure/setup-helm' with ref 'v4', not a pinned commit hash
with:
version: 'latest'

- name: Cache TFlint
uses: actions/cache@v4
with:
path: /home/runner/.tflint.d/plugins
key: ${{ runner.os }}-tflint-${{ hashFiles('.tflint.hcl') }}

- name: Setup TFLint
uses: terraform-linters/setup-tflint@v4

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'terraform-linters/setup-tflint' with ref 'v4', not a pinned commit hash
with:
tflint_version: "v0.51.1"
github_token: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Node only for self-hosted runners
uses: actions/setup-node@v4
if: ${{ inputs.runner_label == 'platform-eng-ent-v2-dual' }}
with:
node-version: 19

- name: Pre-init Hook
run: ${{ inputs.pre_init_hook }}

- name: set PY for precommit cache
run: echo "PY=$(python -VV | sha256sum | cut -d' ' -f1)" >> "$GITHUB_ENV"

- name: Cache Pre-commit
uses: actions/cache@v4
with:
path: ~/.cache/pre-commit
key: pre-commit|${{ env.PY }}|${{ hashFiles('.pre-commit-config.yaml') }}

- name: Precommit Skips
id: precommit_skips
run: |
SKIPS="shellcheck,terraform_fmt,terraform_providers_lock,markdown-link-check,terraform_docs,terraform_tfsec,checkov,terraform_checkov,terraform_checkov"
if [ "${branch}" == "${main_branch}" ];then
SKIPS="${SKIPS},no-commit-to-branch"
fi
if [ "${{inputs.skip_precommit}}" != "" ]; then
SKIPS="${SKIPS},${{inputs.skip_precommit}}"
fi

echo "${SKIPS}"
echo "skips=${SKIPS}" >> "${GITHUB_OUTPUT}"
env:
branch: ${{ github.ref_name }}
main_branch: ${{ inputs.main_branch }}

- name: precommit run tflint hooks for only changed files
id: precommit_run_hooks #Run on all PRs from the same repo, fork repos to run all
if: inputs.pre_commit_run_all == false
continue-on-error: ${{ inputs.skip_tflint_warn_for_changed_files }}
env:
SKIP: ${{ steps.precommit_skips.outputs.skips }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
pip install pre-commit
git fetch origin
if [ "$GITHUB_EVENT_NAME" == 'pull_request' ]
then
if [ ${{ github.event.pull_request.head.repo.full_name }} == ${{ github.repository }} ]
then
pre-commit run --from-ref origin/${{ github.base_ref }} --to-ref ${{ github.event.pull_request.head.sha }} --color=always --show-diff-on-failure
else
pre-commit run --color=always --show-diff-on-failure --all-files
fi
else
pre-commit run --from-ref origin/${{ github.event.repository.default_branch }} --to-ref "$GITHUB_SHA" --color=always --show-diff-on-failure
fi

- name: precommit run tflint hooks
id: precommit_run_hooks_all
if: inputs.pre_commit_run_all
uses: pre-commit/action@v3.0.1

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step: precommit_run_hooks_all
Loading
uses 'pre-commit/action' with ref 'v3.0.1', not a pinned commit hash
continue-on-error: true
env:
SKIP: ${{ steps.precommit_skips.outputs.skips }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
extra_args: --color=always --show-diff-on-failure --all-files

- name: Check status code
if: steps.precommit_run_hooks.outcome == 'failure'
run: |
# SKIP_WARN for skipping all warnings
skip_warn="${{ inputs.skip_tflint_warn }}"
if [ "$skip_warn" = true ]; then
echo "TFLint warnings and errors found and skipped."
else
echo "TFLint errors found, expected '0'. Failing... To enable skipping please set 'skip_tflint_warn' to true in workflow file"
exit 1
fi

#steps.tflint.outcome check for outcome

security_on_dualstack:
name: Security Checks
if: github.ref_name != 'main' && inputs.enable_security_check == true && inputs.runner_label == 'platform-eng-ent-v2-dual'
runs-on:
- ${{ inputs.runner_label }}
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
submodules: ${{ inputs.enable_submodules }}

- name: Run Trivy vulnerability scanner in IaC mode
uses: aquasecurity/trivy-action@0.23.0

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step
Loading
uses 'aquasecurity/trivy-action' with ref '0.23.0', not a pinned commit hash
with:
scan-type: 'config'
hide-progress: false
format: ${{ inputs.trivy_format }}
output: ${{ inputs.trivy_output }}
ignore-unfixed: true
severity: 'CRITICAL,HIGH'

- name: Upload Trivy scan results to Github for inspection
if: ${{ inputs.trivy_inspect_output == 'true' }}
uses: actions/upload-artifact@v4
with:
path: ${{ inputs.trivy_output }}
retention-days: 1

# https://github.com/aquasecurity/trivy/issues/5003
- name: Remove git from url for sarif uploading
if: ${{ inputs.trivy_format == 'sarif' && inputs.trivy_output != '' }}
shell: bash
run: |
sed -i 's#git::https:/##g' ${{ inputs.trivy_output }}

- name: Upload Trivy scan results to GitHub Security tab
if: inputs.upload_sarif == true
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: ${{ inputs.trivy_output }}

- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@v44

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step: changed-files
Loading
uses 'tj-actions/changed-files' with ref 'v44', not a pinned commit hash

- name: Run Checkov action
id: checkov
uses: bridgecrewio/checkov-action@master

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'terraform-ci' step
Uses Step: checkov
Loading
uses 'bridgecrewio/checkov-action' with ref 'master', not a pinned commit hash
with:
output_format: sarif
quiet: ${{ inputs.checkov_output_quiet }}
file: ${{ steps.changed-files.outputs.all_changed_files }}
skip_check: ${{ inputs.checkov_skip_check }}
download_external_modules: ${{ inputs.checkov_download_external_modules }}
skip_path: ${{inputs.checkov_skip_path}}
skip_framework: ${{inputs.checkov_skip_framework}}

- name: Upload Checkov scan results to GitHub Security tab
if: inputs.upload_sarif == true
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'results.sarif'
Loading