Unable to get Action working

[niteshsharoff]

This keeps failing for me when trying to run it in prod. It seems to be related to not finding the correct profile despite me specifying it. Any support would be much appreciated!

Details below:

Errors from Gitub Action under the Run Elementary step:

Notice:  A new release of pip is available: 23.0.1 -> 25.0.1
Notice:  To update, run: pip install --upgrade pip
2025-03-17 08:42:36,862 Setting up the environment.
2025-03-17 08:42:36,863 Getting Elementary *** package version.
2025-03-17 08:42:38,883 Failed to get Elementary *** package version: ***'returncode': 2, 'cmd': ['***', '--log-format', 'json', 'run-operation', 'get_elementary_***_pkg_version', '--project-dir', '/edr_stager_***_project'], 'output': b'***"data": ***"log_version": 3, "version": "=1.9.3"***, "info": ***"category": "", "code": "A001", "extra": ***, "invocation_id": "ff562d90-79f1-4810-92db-7fc06bc1ef2a", "level": "info", "msg": "Running with ***=1.9.3", "name": "MainReportVersion", "pid": 63, "thread": "MainThread", "ts": "2025-03-17T08:42:38.389120Z"***\n***"data": ***"profile_name": "elementary", "target_name": "default"***, "info": ***"category": "", "code": "A005", "extra": ***, "invocation_id": "ff562d90-79f1-4810-92db-7fc06bc1ef2a", "level": "info", "msg": "target not specified in profile \'elementary\', using \'default\'", "name": "MissingProfileTarget", "pid": 63, "thread": "MainThread", "ts": "2025-03-17T08:42:38.394143Z"***\n***"data": ***"exc": "Runtime Error\\n  The profile \'elementary\' does not have a target named \'default\'. The valid target names for this profile are:\\n   - dev\\n   - prod"***, "info": ***"category": "", "code": "Z002", "extra": ***, "invocation_id": "ff562d90-79f1-4810-92db-7fc06bc1ef2a", "level": "error", "msg": "Encountered an error:\\nRuntime Error\\n  The profile \'elementary\' does not have a target named \'default\'. The valid target names for this profile are:\\n   - dev\\n   - prod", "name": "MainEncounteredError", "pid": 63, "thread": "MainThread", "ts": "2025-03-17T08:42:38.394959Z"***\n', 'stderr': b''***
Traceback (most recent call last):
  File "/entrypoint.py", line 158, in <module>
    main()
  File "/entrypoint.py", line 149, in main
    install_edr(args.adapter, args.project_dir, args.profile_target)
  File "/entrypoint.py", line 65, in install_edr
    command_results = subprocess.run(
  File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['***', '--log-format', 'json', 'run-operation', 'get_elementary_***_pkg_version', '--project-dir', '/edr_stager_***_project']' returned non-zero exit status 2.

Github Workflow file:

name: schedule_and_push

on:
  schedule:
    - cron:  '30 9 * * *'
  push:
    branches:
      - main

env:
  DBT_PROFILES_DIR: ./

  DBT_GOOGLE_PROJECT_PROD: ${{ secrets.DBT_GOOGLE_PROJECT_PROD }}
  DBT_GOOGLE_BIGQUERY_DATASET_PROD: ${{ secrets.DBT_GOOGLE_BIGQUERY_DATASET_PROD }}
  # The DBT_GOOGLE_BIGQUERY_KEYFILE_PROD secret will be written to a json file below
  DBT_GOOGLE_BIGQUERY_KEYFILE_PROD: ./dbt-service-account.json

jobs:
  scheduled_run:
    name: scheduled_run
    runs-on: ubuntu-latest

    steps:
      - name: Check out
        uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12.2"

      - name: Authenticate using service account
        shell: bash
        env:
          KEYFILE: ${{secrets.DBT_GOOGLE_BIGQUERY_KEYFILE_PROD}}
        run: |
          echo "$KEYFILE" > ./dbt-service-account.json
      - name: Configure & Install dependencies
        run: |
          python -m pip install --upgrade pip-tools pip wheel
          python -m pip install setuptools==71.1.0 poetry==1.8.3
          pip install pyarrow dbt-core dbt-bigquery elementary_data
          dbt deps --target prod
      - name: Seed dbt
        run: dbt seed --target prod

      # - name: Run dbt models #Commented out until elementary works
      #   run: dbt run --target prod
      
      - name: Test dbt models
        id: dbt_test
        continue-on-error: true
        run: |
          if dbt test --target prod; then
            echo "DBT tests passed"
            echo "test_status=success" >> $GITHUB_OUTPUT
          else
            echo "DBT tests failed"
            echo "test_status=failed" >> $GITHUB_OUTPUT
          fi
      
      # - name: Read profiles.yml
      #   id: profiles
      #   run: |
      #     content=$(cat profiles.yml)
      #     content="${content//'%'/'%25'}"
      #     content="${content//$'\n'/'%0A'}"
      #     content="${content//$'\r'/'%0D'}"
      #     echo "content=$content" >> $GITHUB_OUTPUT
      
      - name: Run Elementary
        if: success() || steps.dbt_test.outcome == 'failure'
        uses: elementary-data/run-elementary-action@v1.12
        with:
          warehouse-type: bigquery
          edr-command: |
            edr send-report 
              --profiles-dir .
              --profile-target prod 
              --env prod 
              --slack-token "https://hooks.slack.com/services/X/Y/Z" 
              --google-service-account-path dbt-service-account.json 
              --gcs-bucket-name XXX
              --bucket-file-path report.html
              --report-url "https://storage.cloud.google.com/XXX/report.html"
            &&
            edr monitor 
              --report-url "https://storage.cloud.google.com/XXX/report.html" 
              --slack-webhook "https://hooks.slack.com/services/X/Y/Z" 
              --profile-target prod 
              --profiles-dir .
              --env prod 
              --group-by table 
              --suppression-interval 0
          bigquery-keyfile: ${{ secrets.DBT_GOOGLE_BIGQUERY_KEYFILE_PROD }}
          gcs-keyfile: ${{ secrets.DBT_GOOGLE_BIGQUERY_KEYFILE_PROD }}

profiles.yml which is in the main root of the DBT project:

dbt:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: service-account
      keyfile: "{{ env_var('DBT_GOOGLE_BIGQUERY_KEYFILE_DEV') }}"
      project: "{{ env_var('DBT_GOOGLE_PROJECT_DEV') }}"
      dataset: "{{ env_var('DBT_GOOGLE_BIGQUERY_DATASET_DEV') }}"
      threads: 32
      location: EU
      timeout_seconds: 300
      priority: interactive

    prod:
      type: bigquery
      method: service-account
      keyfile: "{{ env_var('DBT_GOOGLE_BIGQUERY_KEYFILE_PROD') }}"
      project: "{{ env_var('DBT_GOOGLE_PROJECT_PROD') }}"
      dataset: "{{ env_var('DBT_GOOGLE_BIGQUERY_DATASET_PROD') }}"
      threads: 32
      location: EU
      timeout_seconds: 300
      priority: interactive

elementary:
  outputs:
    dev:
      type: bigquery
      method: service-account
      keyfile: "{{ env_var('DBT_GOOGLE_BIGQUERY_KEYFILE_DEV') }}"
      project: "{{ env_var('DBT_GOOGLE_PROJECT_DEV') }}"
      dataset: "{{ env_var('DBT_GOOGLE_BIGQUERY_DATASET_DEV') }}_elementary"
      threads: 32
      location: EU
      priority: interactive

    prod:
      type: bigquery
      method: service-account
      keyfile: "{{ env_var('DBT_GOOGLE_BIGQUERY_KEYFILE_PROD') }}"
      project: "{{ env_var('DBT_GOOGLE_PROJECT_PROD') }}"
      dataset: "{{ env_var('DBT_GOOGLE_BIGQUERY_DATASET_PROD') }}_elementary"
      threads: 32
      location: EU
      priority: interactive