How to use terraform google provider with Application Default Credentials

[jguay]

How can I use google terraform provider with Application Default Credentials (as opposed to service account credentials) ?

Dockerfile :

FROM semaphoreui/semaphore:latest
RUN if [[ $(uname -m) == "aarch64" ]]; then cd ~ && curl -sOL https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-arm.tar.gz && tar -xf google-cloud-cli-linux-arm.tar.gz && google-cloud-sdk/install.sh -q && rm google-cloud-cli-linux-arm.tar.gz; else cd ~ && curl -sOL https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-linux-x86_64.tar.gz && tar -xf google-cloud-cli-linux-x86_64.tar.gz && google-cloud-sdk/install.sh -q && rm google-cloud-cli-linux-x86_64.tar.gz; fi
ENV PATH="${PATH}:/home/semaphore/google-cloud-sdk/bin"

docker exec -it semaphore01 /bin/bash :

semaphore01:~$ semaphore version
v2.10.32-f33944e-1729509436
semaphore01:~$ gcloud auth application-default login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fapplicationdefaultauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=ASSne1mbvCaFhLgVekj5Kgu2urNVHd&prompt=consent&token_usage=remote&access_type=offline&code_challenge=ncJtctqy9mLcfSI7VdgPTkKTkH8gDZLq3xsTsTsmSSI&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: <REDACTED>

Credentials saved to file: [/home/semaphore/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).
WARNING:
Cannot find a quota project to add to ADC. You might receive a "quota exceeded" or "API not enabled" error. Run $ gcloud auth application-default set-quota-project to add a quota project.

After this running a terraform main.tf :

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "~> 3.0"
    }
  }
}

provider "google" {
  project = "<REDACTED>"
  region  = "europe-west2"
  zone    = "europe-west2-c"
}

resource "google_compute_instance" "default" {
  name         = "test"
  machine_type = "n1-standard-1"
  zone         = "us-central1-a"

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-11"
    }
  }

  network_interface {
    network = "default"
    access_config {
      // Ephemeral IP
    }
  }
}

I can use inside the container terraform init && terraform apply -auto-approve from a temp folder with the file above and the instance is created in GCE (terraform destroy -auto-aprove also delete the VM as expected)

The output from semaphore task template run :

4:16:30 PM Planning failed. Terraform encountered an error while generating this plan.
4:16:30 PM 4:16:30 PM ╷
4:16:30 PM │ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'.  Original error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
4:16:30 PM │ 
4:16:30 PM │   with provider["registry.terraform.io/hashicorp/google"],
4:16:30 PM │   on main.tf line 10, in provider "google":
4:16:30 PM │   10: provider "google" {
4:16:30 PM │ 
4:16:30 PM ╵
4:16:30 PM 4:16:30 PM Running app failed: exit status 1

I am wondering under which user terraform is executed if this is not semaphore user or why terraform run by semaphore cannot find credendials that work as interactive user

As a note - using a service account works for me (using a volume to pass the service json account) - I am trying to see if users who don't have permissions to create a service account with required roles can just use their own GCP account

[jguay]

Using bash I saw HOME=/tmp/semaphore which might explain the issue I had (I got gcloud to use my docker host credentials by passing its config to that home directory with a volume ~/.config/gcloud:/tmp/semaphore/.config/gcloud - will test if terraform uses that and report back)