Skip to content

Add dags and plugins to staging composer #3833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Apr 15, 2025
Merged

Add dags and plugins to staging composer #3833

merged 7 commits into from
Apr 15, 2025

Conversation

ohrite
Copy link
Contributor

@ohrite ohrite commented Apr 15, 2025

Description

Adds a new deployment mechanism for staging Airflow DAGs and plugins: use terraform to control the bucket contents.

Part of #3765

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

terraform plan

Post-merge follow-ups

  • No action required
  • Actions required (specified below)

Monitor main post-merge

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 15, 2025 •
edited
Loading

Terraform plan in iac/cal-itp-data-infra-staging/gcs/us

Plan: 1 to add, 1 to change, 0 to destroy. 
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create
!~  update in-place

Terraform will perform the following actions:

  # google_storage_bucket.calitp-staging-composer will be updated in-place
!~  resource "google_storage_bucket" "calitp-staging-composer" {
        id                          = "calitp-staging-composer"
        name                        = "calitp-staging-composer"
!~      terraform_labels            = {
+           "goog-composer-environment"      = "calitp-staging-composer"
+           "goog-composer-environment-uuid" = "23aa69a2-14bb-410f-ba28-ad96dd552748"
+           "goog-composer-location"         = "us-west2"
+           "goog-composer-version"          = "composer-2-8-3-airflow-2-6-3"
+           "goog-terraform-provisioned"     = "true"
        }
#        (14 unchanged attributes hidden)

#        (2 unchanged blocks hidden)
    }

  # google_storage_bucket_iam_member.calitp-staging-composer will be created
+   resource "google_storage_bucket_iam_member" "calitp-staging-composer" {
+       bucket = "calitp-staging-composer"
+       etag   = (known after apply)
+       id     = (known after apply)
+       member = "projectEditor:cal-itp-data-infra-staging"
+       role   = "roles/storage.legacyBucketOwner"
    }

Plan: 1 to add, 1 to change, 0 to destroy.

📝 Plan generated in Terraform Plan #98

@ohrite ohrite force-pushed the mov/3765-terraform-staging-composer-dags branch 2 times, most recently from 3a62822 to aca203b Compare April 15, 2025 04:10
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 15, 2025 •
edited
Loading

Terraform plan in iac/cal-itp-data-infra-staging/airflow/us

Plan: 403 to add, 0 to change, 0 to destroy. 
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+   create

Terraform will perform the following actions:

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/METADATA.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/METADATA.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/METADATA.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/README.md"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/README.md"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/README.md"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_county_geography.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_county_geography.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_county_geography.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_eligibility_programs.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_eligibility_programs.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_eligibility_programs.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_fare_systems.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_fare_systems.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_fare_systems.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_funding_programs.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_funding_programs.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_funding_programs.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_gtfs_datasets.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_gtfs_datasets.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_gtfs_datasets.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_gtfs_service_data.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_gtfs_service_data.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_gtfs_service_data.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_modes.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_modes.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_modes.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_ntd_agency_info.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_ntd_agency_info.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_ntd_agency_info.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_organizations.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_organizations.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_organizations.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_place_geography.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_place_geography.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_place_geography.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_rider_requirements.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_rider_requirements.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_rider_requirements.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/california_transit_services.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/california_transit_services.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/california_transit_services.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/generate_gtfs_download_configs.py"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/generate_gtfs_download_configs.py"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/generate_gtfs_download_configs.py"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_data_quality_issues_gtfs_datasets.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_data_quality_issues_gtfs_datasets.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_data_quality_issues_gtfs_datasets.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_data_quality_issues_issue_types.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_data_quality_issues_issue_types.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_data_quality_issues_issue_types.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_data_quality_issues_services.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_data_quality_issues_services.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_data_quality_issues_services.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_data_quality_issues_transit_data_quality_issues.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_data_quality_issues_transit_data_quality_issues.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_data_quality_issues_transit_data_quality_issues.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_components.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_components.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_components.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_contracts.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_contracts.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_contracts.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_data_schemas.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_data_schemas.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_data_schemas.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_organizations.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_organizations.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_organizations.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_products.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_products.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_products.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_properties_features.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_properties_features.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_properties_features.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_relationships_service_components.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_relationships_service_components.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_relationships_service_components.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_service_components.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_service_components.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_service_components.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/airtable_loader_v2/transit_tech_stacks_services.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/airtable_loader_v2/transit_tech_stacks_services.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/airtable_loader_v2/transit_tech_stacks_services.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/copy_prod_archiver_configs_to_test/METADATA.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/copy_prod_archiver_configs_to_test/METADATA.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/copy_prod_archiver_configs_to_test/METADATA.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/copy_prod_archiver_configs_to_test/README.md"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/copy_prod_archiver_configs_to_test/README.md"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/copy_prod_archiver_configs_to_test/README.md"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/copy_prod_archiver_configs_to_test/copy_prod_archiver_configs_to_test.py"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/copy_prod_archiver_configs_to_test/copy_prod_archiver_configs_to_test.py"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/copy_prod_archiver_configs_to_test/copy_prod_archiver_configs_to_test.py"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/METADATA.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/METADATA.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/METADATA.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/README.md"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/README.md"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/README.md"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/METADATA.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/METADATA.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/METADATA.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_county_geography.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_county_geography.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_county_geography.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_eligibility_programs.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_eligibility_programs.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_eligibility_programs.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_fare_systems.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_fare_systems.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_fare_systems.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_funding_programs.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_funding_programs.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_funding_programs.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_datasets.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_datasets.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_datasets.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_service_data.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_service_data.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_gtfs_service_data.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_modes.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_modes.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_modes.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_ntd_agency_info.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_ntd_agency_info.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_ntd_agency_info.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_organizations.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_organizations.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_organizations.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_place_geography.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_place_geography.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_place_geography.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_rider_requirements.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_rider_requirements.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_rider_requirements.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_california_transit_services.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_california_transit_services.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_california_transit_services.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_gtfs_datasets.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_gtfs_datasets.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_gtfs_datasets.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_issue_types.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_issue_types.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_issue_types.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_services.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_services.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_services.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_transit_data_quality_issues.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_transit_data_quality_issues.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_data_quality_issues_transit_data_quality_issues.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_components.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_components.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_components.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_contracts.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_contracts.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_contracts.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_data_schemas.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_data_schemas.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
+       source         = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_data_schemas.yml"
+       storage_class  = (known after apply)
    }

  # google_storage_bucket_object.calitp-staging-composer["dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_organizations.yml"] will be created
+   resource "google_storage_bucket_object" "calitp-staging-composer" {
+       bucket         = "calitp-staging-composer"
+       content_type   = (known after apply)
+       crc32c         = (known after apply)
+       detect_md5hash = "different hash"
+       id             = (known after apply)
+       kms_key_name   = (known after apply)
+       md5hash        = (known after apply)
+       media_link     = (known after apply)
+       name           = "dags/create_external_tables/airtable/external_airtable_transit_tech_stacks_organizations.yml"
+       output_name    = (known after apply)
+       self_link      = (known after apply)
Plan is too large to fit in a PR comment. See the full plan in the workflow log.

📝 Plan generated in Terraform Plan #98

@ohrite ohrite force-pushed the mov/3765-terraform-staging-composer-dags branch 2 times, most recently from 74591c3 to 1ec1c15 Compare April 15, 2025 04:18
@ohrite ohrite force-pushed the mov/3765-terraform-staging-composer-dags branch from 1ec1c15 to c487d15 Compare April 15, 2025 04:22
@ohrite ohrite force-pushed the mov/3765-terraform-staging-composer-dags branch from c487d15 to 88e7c31 Compare April 15, 2025 16:56
@ohrite ohrite self-assigned this Apr 15, 2025
@ohrite ohrite requested a review from erikamov April 15, 2025 16:56
erikamov
erikamov approved these changes Apr 15, 2025
View reviewed changes
Copy link
Contributor

@erikamov erikamov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work.

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 15, 2025 •
edited
Loading

Terraform plan in iac/cal-itp-data-infra/gcs/us

Plan: 0 to add, 6 to change, 0 to destroy. 
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~  update in-place

Terraform will perform the following actions:

  # google_storage_bucket.tfer--calitp-gtfs-rt-raw-deprecated will be updated in-place
!~  resource "google_storage_bucket" "tfer--calitp-gtfs-rt-raw-deprecated" {
        id                          = "calitp-gtfs-rt-raw-deprecated"
        name                        = "calitp-gtfs-rt-raw-deprecated"
!~      terraform_labels            = {
+           "deprecated" = "true"
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (2 unchanged blocks hidden)
    }

  # google_storage_bucket.tfer--dataproc-temp-us-west2-1005246706141-x9mtxbwg will be updated in-place
!~  resource "google_storage_bucket" "tfer--dataproc-temp-us-west2-1005246706141-x9mtxbwg" {
        id                          = "dataproc-temp-us-west2-1005246706141-x9mtxbwg"
        name                        = "dataproc-temp-us-west2-1005246706141-x9mtxbwg"
!~      terraform_labels            = {
+           "goog-dataproc-location" = "us-west2"
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (3 unchanged blocks hidden)
    }

  # google_storage_bucket.tfer--rt-parsed-deprecated will be updated in-place
!~  resource "google_storage_bucket" "tfer--rt-parsed-deprecated" {
        id                          = "rt-parsed-deprecated"
        name                        = "rt-parsed-deprecated"
!~      terraform_labels            = {
+           "deprecated" = (known after apply)
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (2 unchanged blocks hidden)
    }

  # google_storage_bucket.tfer--us-west2-calitp-airflow2-pr-171e4e47-bucket will be updated in-place
!~  resource "google_storage_bucket" "tfer--us-west2-calitp-airflow2-pr-171e4e47-bucket" {
        id                          = "us-west2-calitp-airflow2-pr-171e4e47-bucket"
        name                        = "us-west2-calitp-airflow2-pr-171e4e47-bucket"
!~      terraform_labels            = {
+           "goog-composer-environment"         = "calitp-airflow2-prod"
+           "goog-composer-environment-deleted" = "true"
+           "goog-composer-environment-uuid"    = "8731c3aa-7c40-4da7-9bf6-6aa1759ec2ec"
+           "goog-composer-location"            = "us-west2"
+           "goog-composer-version"             = "composer-1-20-6-airflow-2-4-3"
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (2 unchanged blocks hidden)
    }

  # google_storage_bucket.tfer--us-west2-calitp-airflow2-pr-31e41084-bucket will be updated in-place
!~  resource "google_storage_bucket" "tfer--us-west2-calitp-airflow2-pr-31e41084-bucket" {
        id                          = "us-west2-calitp-airflow2-pr-31e41084-bucket"
        name                        = "us-west2-calitp-airflow2-pr-31e41084-bucket"
!~      terraform_labels            = {
+           "goog-composer-environment"         = "calitp-airflow2-prod-composer2"
+           "goog-composer-environment-deleted" = "true"
+           "goog-composer-environment-uuid"    = "43d3f4a4-235d-4f04-addc-6d86e6542b09"
+           "goog-composer-location"            = "us-west2"
+           "goog-composer-version"             = "composer-2-6-0-airflow-2-5-3"
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (2 unchanged blocks hidden)
    }

  # google_storage_bucket.tfer--us-west2-calitp-airflow2-pr-88ca8ec6-bucket will be updated in-place
!~  resource "google_storage_bucket" "tfer--us-west2-calitp-airflow2-pr-88ca8ec6-bucket" {
        id                          = "us-west2-calitp-airflow2-pr-88ca8ec6-bucket"
        name                        = "us-west2-calitp-airflow2-pr-88ca8ec6-bucket"
!~      terraform_labels            = {
+           "goog-composer-environment"      = "calitp-airflow2-prod-composer2-patch"
+           "goog-composer-environment-uuid" = "b2062ffc-a77d-44a2-8b4e-05f8f5bf4996"
+           "goog-composer-location"         = "us-west2"
+           "goog-composer-version"          = "composer-2-6-0-airflow-2-5-3"
        }
#        (14 unchanged attributes hidden)

-       timeouts {}

#        (2 unchanged blocks hidden)
    }

Plan: 0 to add, 6 to change, 0 to destroy.

📝 Plan generated in Terraform Plan #98

@ohrite ohrite merged commit f870d11 into main Apr 15, 2025
5 checks passed
@ohrite ohrite deleted the mov/3765-terraform-staging-composer-dags branch April 15, 2025 20:57
@ohrite ohrite mentioned this pull request Apr 16, 2025
Merged
6 tasks
@vevetron
Copy link
Contributor

Very cool.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@erikamov erikamov erikamov approved these changes

@evansiroky evansiroky Awaiting requested review from evansiroky evansiroky is a code owner

@vevetron vevetron Awaiting requested review from vevetron vevetron is a code owner

Assignees

@ohrite ohrite

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ohrite @vevetron @erikamov