Unable to create external email notifications on dbt Cloud terraform provider 1.2.2

[sanjay-lendi]

Describe the bug

Getting a plugin error when creating dbtcloud_notification using the latest terraform provider 1.2.2 with notification_type = 4

Error message

Here is the buildkite logs showing the plugin error.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:

  + create

  ~ update in-place



Terraform will perform the following actions:



  # dbtcloud_job.dev_job_full_refresh will be updated in-place

  ~ resource "dbtcloud_job" "dev_job_full_refresh" {

        id                     = 111111

        name                   = "Manual Full Refresh - DEV"

      - schedule_days          = [

          - 0,

          - 1,

          - 2,

          - 3,

          - 4,

          - 5,

          - 6,

        ] -> null

      ~ schedule_type          = "days_of_week" -> "every_day"

      ~ self_deferring         = false -> (known after apply)

        # (19 unchanged attributes hidden)

    }



  # dbtcloud_notification.dev_jobs_email_notification will be created

  + resource "dbtcloud_notification" "dev_jobs_email_notification" {

      + external_email    = "channel@slack.com"

      + id                = (known after apply)

      + notification_type = 4

      + on_cancel         = [

          + 123456,

          + 123457,

        ]

      + on_failure        = [

          + 123456,

          + 123457,

        ]

      + on_success        = []

      + on_warning        = []

      + state             = 1

      + user_id           = 3333

    }



  # dbtcloud_notification.production_jobs_email_notification will be created

  + resource "dbtcloud_notification" "production_jobs_email_notification" {

      + external_email    = "channel@slack.com"

      + id                = (known after apply)

      + notification_type = 4

      + on_cancel         = [

          + 223456,

          + 223457,

        ]

      + on_failure        = [

          + 223456,

          + 223457,

        ]

      + on_success        = []

      + on_warning        = []

      + state             = 1

      + user_id           = 3333

    }



  # dbtcloud_notification.staging_jobs_email_notification will be created

  + resource "dbtcloud_notification" "staging_jobs_email_notification" {

      + external_email    = "channel@slack.com"

      + id                = (known after apply)

      + notification_type = 4

      + on_cancel         = [

          + 33348,

          + 33349,

        ]

      + on_failure        = [

          + 33348,

          + 33349,

        ]

      + on_success        = []

      + on_warning        = []

      + state             = 1

      + user_id           = 3333

    }



  # module.dbtcloud_environments.dbtcloud_environment.local["local"] will be updated in-place

  ~ resource "dbtcloud_environment" "local" {

      + credential_id              = (known after apply)

      + extended_attributes_id     = (known after apply)

        id                         = "11111:22222"

        name                       = "local"

        # (9 unchanged attributes hidden)

    }



  # module.dbtcloud_environments.dbtcloud_environment.this["dev"] will be updated in-place

  ~ resource "dbtcloud_environment" "this" {

      + extended_attributes_id     = (known after apply)

        id                         = "11111:22222"

        name                       = "dev"

        # (11 unchanged attributes hidden)

    }



Plan: 3 to add, 3 to change, 0 to destroy.



Warning: Deprecated attribute



  on locals.tf line 2, in locals:

   2:   region         = data.aws_region.current.name



The attribute "name" is deprecated. Refer to the provider documentation for details.



(and one more similar warning elsewhere)



───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────



Saved the plan to: /tmp/terraform-dbt-cloud-development.json



To perform exactly these actions, run the following command to apply:

    terraform apply "/tmp/terraform-dbt-cloud-development.json"

👤 Assuming the role...

(node:463) NOTE: The AWS SDK for JavaScript (v2) is in maintenance mode.

 SDK releases are limited to address critical bug fixes and security issues only.



Please migrate your code to use AWS SDK for JavaScript (v3).

For more information, check the blog post at https://a.co/cUPnyil

(Use `node --trace-warnings ...` to show where the warning was created)

Starting LTD Terraform Apply...

Environment: development

Project: dbt-cloud

module.dbtcloud_environments.dbtcloud_environment.this["dev"]: Modifying... [id=13091:27108]

module.dbtcloud_environments.dbtcloud_environment.local["local"]: Modifying... [id=13091:28661]

 module.dbtcloud_environments.dbtcloud_environment.local["local"]: Modifications complete after 0s [id=13091:28661] 

 module.dbtcloud_environments.dbtcloud_environment.this["dev"]: Modifications complete after 0s [id=13091:27108] 

 dbtcloud_notification.staging_jobs_email_notification: Creating...

 dbtcloud_notification.production_jobs_email_notification: Creating...

 dbtcloud_job.dev_job_full_refresh: Modifying... [name=Manual Full Refresh - DEV]

 │    Error:   Request cancelled 

 │   

 │  The plugin6.(*GRPCProvider).ApplyResourceChange request was cancelled.

 ╵

 ╷

 │    Error:   Plugin did not respond 

 │   

 │  The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ApplyResourceChange call. The plugin logs may contain more details.

 ╵

 ╷

 │    Error:   Request cancelled 

 │   

 │  The plugin6.(*GRPCProvider).ApplyResourceChange request was cancelled.

 ╵



 Stack trace from the terraform-provider-dbtcloud_v1.2.2 plugin:



 panic: runtime error: invalid memory address or nil pointer dereference

 [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xa060c8]



 goroutine 90 [running]:

  github.com/dbt-labs/terraform-provider-dbtcloud/pkg/framework/objects/notification.(*notificationResource).Create(0xc0001146f0, {0x10751c8, 0xc00016bd40}, {{{{0x107b290, 0xc000380150}, {0xdd8bc0, 0xc000661950}}, {0x1080310, 0xc0005003c0}}, {{{0x107b290, ...}, ...}, ...}, ...}, ...)

	 github.com/dbt-labs/terraform-provider-dbtcloud/pkg/framework/objects/notification/resource.go:189 +0x5e8

  github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).CreateResource(0xc000175380, {0x10751c8, 0xc00016bd40}, 0xc00053d600, 0xc00053d5d8)

	 github.com/hashicorp/terraform-plugin-framework@v1.8.0/internal/fwserver/server_createresource.go:101 +0x577

 github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ApplyResourceChange(0xc000175380, {0x10751c8, 0xc00016bd40}, 0xc0005fbd60, 0xc00053d6f0)

	 github.com/hashicorp/terraform-plugin-framework@v1.8.0/internal/fwserver/server_applyresourcechange.go:57 +0x4aa

 github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ApplyResourceChange(0xc000175380, {0x10751c8?, 0xc00016bb90?}, 0xc0005fbcc0)

	 github.com/hashicorp/terraform-plugin-framework@v1.8.0/internal/proto6server/server_applyresourcechange.go:55 +0x385

 github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ApplyResourceChange(0xc0002ddcc0, {0x10751c8?, 0xc00016a570?}, 0xc00028a150)

	 github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov6/tf6server/server.go:865 +0x3b9

 github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ApplyResourceChange_Handler({0xebcc60, 0xc0002ddcc0}, {0x10751c8, 0xc00016a570}, 0xc0007db880, 0x0)

	 github.com/hashicorp/terraform-plugin-go@v0.23.0/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:518 +0x1a6

 google.golang.org/grpc.(*Server).processUnaryRPC(0xc000223000, {0x10751c8, 0xc000701f50}, {0x107e020, 0xc0002fe300}, 0xc00047b320, 0xc000456360, 0x17be6f8, 0x0)

	 google.golang.org/grpc@v1.63.2/server.go:1369 +0xde3

 google.golang.org/grpc.(*Server).handleStream(0xc000223000, {0x107e020, 0xc0002fe300}, 0xc00047b320)

	 google.golang.org/grpc@v1.63.2/server.go:1780 +0xe8a

 google.golang.org/grpc.(*Server).serveStreams.func2.1()

	 google.golang.org/grpc@v1.63.2/server.go:1019 +0x7f

 created by  google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 41

	 google.golang.org/grpc@v1.63.2/server.go:1030 +0x11d



 Error: The terraform-provider-dbtcloud_v1.2.2 plugin crashed!



 This is always indicative of a bug within the plugin. It would be immensely

 helpful if you could report the crash with the plugin's maintainers so that it

 can be fixed. The output above should help diagnose the issue.



 ^^^ +++
 🚨 Error: The command exited with status 1 
 ^^^ +++
 user command error: running "plugin docker command" shell hook: The plugin docker command hook exited with status 1

Resource configuration

The relevant resource configuration (sanitised so it doesn't contain sensitive data)

resource "dbtcloud_notification" "dev_jobs_email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel@slack.com"
}

resource "dbtcloud_notification" "staging_jobs_email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.staging_job.id, dbtcloud_job.staging_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.staging_job.id, dbtcloud_job.staging_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel@slack.com"
}

resource "dbtcloud_notification" "production_jobs_email_notification" {
  user_id    = 3966
  on_failure = [dbtcloud_job.production_job.id, dbtcloud_job.production_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.production_job.id, dbtcloud_job.production_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel@slack.com"
}

Expected behavior

No errors and a successful deployment of notifications.

Config (please complete the following information):

( the version can be retrieved running the command terraform providers )

• dbtcloud 1.2.2

Additional context

Add any other context about the problem here.

[adrianburusdbt]

Hey Sanjay, could you please provide more details about your configuration? I tried setting a type notification on a few jobs and it works for me, but what strikes me as odd in your configuration is that you have multiple resources pointing to the same external_email within the same account.
dbt cloud has a constraint that the (external_email, account) pair is unique, so I would expect this config to fail with a message like this: Key (account_id, external_email)=(XXXXX, XXX@XXXX.com) already exists.
Also, the user_id is basically the user that's creating the email notification. So in the UI, it would be the user that's actually creating the config, implicitly set. In the provider, we should use a single user_id, which you can get from dbt-cloud directly.

So a valid configuration, based on your example above, looks like this:

resource "dbtcloud_notification" "email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id, dbtcloud_job.staging_job.id, dbtcloud_job.staging_job_full_refresh.id, dbtcloud_job.production_job.id, dbtcloud_job.production_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id, dbtcloud_job.staging_job_full_refresh.id, dbtcloud_job.staging_job.id, dbtcloud_job.staging_job_full_refresh.id, dbtcloud_job.production_job.id, dbtcloud_job.production_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel@slack.com"
}

Let me know if you need more assistance.

[sanjay-lendi]

Yeah apologies, my bad on not properly masking the email addresses. So we have a requirement for notifying different slack channels based on the environment, so dev alerts go to channel1@slack.com email, staging to channel2@slack.com and prod to channel3@slack.com email address.

My config looks like something like this:

resource "dbtcloud_notification" "dev_email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.dev_job.id, dbtcloud_job.dev_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel1@slack.com"
}

resource "dbtcloud_notification" "staging_email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.stg_job.id, dbtcloud_job.stg_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.stg_job.id, dbtcloud_job.stg_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel2@slack.com"
}

resource "dbtcloud_notification" "prod_email_notification" {
  user_id    = 1234
  on_failure = [dbtcloud_job.prod_job.id, dbtcloud_job.prod_job_full_refresh.id]
  on_cancel  = [dbtcloud_job.prod_job.id, dbtcloud_job.prod_job_full_refresh.id]
  // the Type 4 is used for external notifications
  notification_type = 4
  // the external_email is the email address that will receive the notification
  external_email = "channel3@slack.com"
}

Is this possible and an allowed configuration ? I am getting errors for this config as shown in the original post.