feat: Module updates:<br> - resource_tags input variable renamed to tags<br> - cbr_rules has been extended to support tags and operations<br> * DA updates: <br> - Added support for passing existing Postgresql instance <br> - Added a new input variable existing_postgresql_instance_crn (#596)

* test-coverage:postgresql

* fix:upgrade-test

* added variables in catalog.json

* fixed:pre-commit

---------

Co-authored-by: Aatreyee Mukherjee <aatreyeemukherjee@Aatreyees-MacBook-Pro.local>

Author: aatreyee2506

README.md

@@ -98,7 +98,7 @@ To attach access management tags to resources in this module, you need the follo
 | <a name="input_auto_scaling"></a> [auto\_scaling](#input\_auto\_scaling) | Optional rules to allow the database to increase resources in response to usage. Only a single autoscaling block is allowed. Make sure you understand the effects of autoscaling, especially for production environments. See https://ibm.biz/autoscaling-considerations in the IBM Cloud Docs. | <pre>object({<br/>    disk = object({<br/>      capacity_enabled             = optional(bool, false)<br/>      free_space_less_than_percent = optional(number, 10)<br/>      io_above_percent             = optional(number, 90)<br/>      io_enabled                   = optional(bool, false)<br/>      io_over_period               = optional(string, "15m")<br/>      rate_increase_percent        = optional(number, 10)<br/>      rate_limit_mb_per_member     = optional(number, 3670016)<br/>      rate_period_seconds          = optional(number, 900)<br/>      rate_units                   = optional(string, "mb")<br/>    })<br/>    memory = object({<br/>      io_above_percent         = optional(number, 90)<br/>      io_enabled               = optional(bool, false)<br/>      io_over_period           = optional(string, "15m")<br/>      rate_increase_percent    = optional(number, 10)<br/>      rate_limit_mb_per_member = optional(number, 114688)<br/>      rate_period_seconds      = optional(number, 900)<br/>      rate_units               = optional(string, "mb")<br/>    })<br/>  })</pre> | `null` | no |
 | <a name="input_backup_crn"></a> [backup\_crn](#input\_backup\_crn) | The CRN of a backup resource to restore from. The backup is created by a database deployment with the same service ID. The backup is loaded after provisioning and the new deployment starts up that uses that data. A backup CRN is in the format crn:v1:<…>:backup:. If omitted, the database is provisioned empty. | `string` | `null` | no |
 | <a name="input_backup_encryption_key_crn"></a> [backup\_encryption\_key\_crn](#input\_backup\_encryption\_key\_crn) | The CRN of a Key Protect or Hyper Protect Crypto Services encryption key that you want to use for encrypting the disk that holds deployment backups. Applies only if `use_ibm_owned_encryption_key` is false and `use_same_kms_key_for_backups` is false. If no value is passed, and `use_same_kms_key_for_backups` is true, the value of `kms_key_crn` is used. Alternatively set `use_default_backup_encryption_key` to true to use the IBM Cloud Databases default encryption. Bare in mind that backups encryption is only available in certain regions. See [Bring your own key for backups](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect&interface=ui#key-byok) and [Using the HPCS Key for Backup encryption](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs#use-hpcs-backups). | `string` | `null` | no |
-| <a name="input_cbr_rules"></a> [cbr\_rules](#input\_cbr\_rules) | (Optional, list) List of CBR rules to create | <pre>list(object({<br/>    description = string<br/>    account_id  = string<br/>    rule_contexts = list(object({<br/>      attributes = optional(list(object({<br/>        name  = string<br/>        value = string<br/>    }))) }))<br/>    enforcement_mode = string<br/>  }))</pre> | `[]` | no |
+| <a name="input_cbr_rules"></a> [cbr\_rules](#input\_cbr\_rules) | (Optional, list) List of CBR rules to create | <pre>list(object({<br/>    description = string<br/>    account_id  = string<br/>    rule_contexts = list(object({<br/>      attributes = optional(list(object({<br/>        name  = string<br/>        value = string<br/>    }))) }))<br/>    enforcement_mode = string<br/>    tags = optional(list(object({<br/>      name  = string<br/>      value = string<br/>    })))<br/>  }))</pre> | `[]` | no |
 | <a name="input_configuration"></a> [configuration](#input\_configuration) | Database configuration parameters, see https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-changing-configuration&interface=api for more details. | <pre>object({<br/>    shared_buffers  = optional(number)<br/>    max_connections = optional(number)<br/>    # below field gives error when sent to provider<br/>    # tracking issue: https://github.com/IBM-Cloud/terraform-provider-ibm/issues/5403<br/>    # max_locks_per_transaction  = optional(number)<br/>    max_prepared_transactions  = optional(number)<br/>    synchronous_commit         = optional(string)<br/>    effective_io_concurrency   = optional(number)<br/>    deadlock_timeout           = optional(number)<br/>    log_connections            = optional(string)<br/>    log_disconnections         = optional(string)<br/>    log_min_duration_statement = optional(number)<br/>    tcp_keepalives_idle        = optional(number)<br/>    tcp_keepalives_interval    = optional(number)<br/>    tcp_keepalives_count       = optional(number)<br/>    archive_timeout            = optional(number)<br/>    wal_level                  = optional(string)<br/>    max_replication_slots      = optional(number)<br/>    max_wal_senders            = optional(number)<br/>  })</pre> | `null` | no |
 | <a name="input_kms_key_crn"></a> [kms\_key\_crn](#input\_kms\_key\_crn) | The CRN of a Key Protect or Hyper Protect Crypto Services encryption key to encrypt your data. Applies only if `use_ibm_owned_encryption_key` is false. By default this key is used for both deployment data and backups, but this behaviour can be altered using the `use_same_kms_key_for_backups` and `backup_encryption_key_crn` inputs. Bare in mind that backups encryption is only available in certain regions. See [Bring your own key for backups](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect&interface=ui#key-byok) and [Using the HPCS Key for Backup encryption](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs#use-hpcs-backups). | `string` | `null` | no |
 | <a name="input_member_cpu_count"></a> [member\_cpu\_count](#input\_member\_cpu\_count) | Allocated dedicated CPU per member. For shared CPU, set to 0. [Learn more](https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-resources-scaling). Ignored during restore and point in time recovery operations | `number` | `0` | no |
@@ -113,10 +113,10 @@ To attach access management tags to resources in this module, you need the follo
 | <a name="input_region"></a> [region](#input\_region) | The region where you want to deploy your instance. | `string` | `"us-south"` | no |
 | <a name="input_remote_leader_crn"></a> [remote\_leader\_crn](#input\_remote\_leader\_crn) | A CRN of the leader database to make the replica(read-only) deployment. The leader database is created by a database deployment with the same service ID. A read-only replica is set up to replicate all of your data from the leader deployment to the replica deployment by using asynchronous replication. For more information, see https://cloud.ibm.com/docs/databases-for-postgresql?topic=databases-for-postgresql-read-only-replicas | `string` | `null` | no |
 | <a name="input_resource_group_id"></a> [resource\_group\_id](#input\_resource\_group\_id) | The resource group ID where the PostgreSQL instance will be created. | `string` | n/a | yes |
-| <a name="input_resource_tags"></a> [resource\_tags](#input\_resource\_tags) | Optional list of tags to be added to the PostgreSQL instance. | `list(string)` | `[]` | no |
 | <a name="input_service_credential_names"></a> [service\_credential\_names](#input\_service\_credential\_names) | Map of name, role for service credentials that you want to create for the database | `map(string)` | `{}` | no |
 | <a name="input_service_endpoints"></a> [service\_endpoints](#input\_service\_endpoints) | Specify whether you want to enable the public, private, or both service endpoints. Supported values are 'public', 'private', or 'public-and-private'. | `string` | `"private"` | no |
 | <a name="input_skip_iam_authorization_policy"></a> [skip\_iam\_authorization\_policy](#input\_skip\_iam\_authorization\_policy) | Set to true to skip the creation of IAM authorization policies that permits all Databases for PostgreSQL instances in the given resource group 'Reader' access to the Key Protect or Hyper Protect Crypto Services key that was provided in the `kms_key_crn` and `backup_encryption_key_crn` inputs. This policy is required in order to enable KMS encryption, so only skip creation if there is one already present in your account. No policy is created if `use_ibm_owned_encryption_key` is true. | `bool` | `false` | no |
+| <a name="input_tags"></a> [tags](#input\_tags) | Optional list of tags to be added to the PostgreSQL instance. | `list(string)` | `[]` | no |
 | <a name="input_use_default_backup_encryption_key"></a> [use\_default\_backup\_encryption\_key](#input\_use\_default\_backup\_encryption\_key) | When `use_ibm_owned_encryption_key` is set to false, backups will be encrypted with either the key specified in `kms_key_crn`, or in `backup_encryption_key_crn` if a value is passed. If you do not want to use your own key for backups encryption, you can set this to `true` to use the IBM Cloud Databases default encryption for backups. Alternatively set `use_ibm_owned_encryption_key` to true to use the default encryption for both backups and deployment data. | `bool` | `false` | no |
 | <a name="input_use_ibm_owned_encryption_key"></a> [use\_ibm\_owned\_encryption\_key](#input\_use\_ibm\_owned\_encryption\_key) | IBM Cloud Databases will secure your deployment's data at rest automatically with an encryption key that IBM hold. Alternatively, you may select your own Key Management System instance and encryption key (Key Protect or Hyper Protect Crypto Services) by setting this to false. If setting to false, a value must be passed for the `kms_key_crn` input. | `bool` | `true` | no |
 | <a name="input_use_same_kms_key_for_backups"></a> [use\_same\_kms\_key\_for\_backups](#input\_use\_same\_kms\_key\_for\_backups) | Set this to false if you wan't to use a different key that you own to encrypt backups. When set to false, a value is required for the `backup_encryption_key_crn` input. Alternatiely set `use_default_backup_encryption_key` to true to use the IBM Cloud Databases default encryption. Applies only if `use_ibm_owned_encryption_key` is false. Bare in mind that backups encryption is only available in certain regions. See [Bring your own key for backups](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect&interface=ui#key-byok) and [Using the HPCS Key for Backup encryption](https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs#use-hpcs-backups). | `bool` | `true` | no |

examples/backup/main.tf

@@ -17,7 +17,7 @@ module "postgresql_db" {
   name               = "${var.prefix}-postgres"
   pg_version         = var.pg_version
   region             = var.region
-  resource_tags      = var.resource_tags
+  tags               = var.resource_tags
   access_tags        = var.access_tags
   member_host_flavor = "multitenant"
 }
@@ -28,13 +28,13 @@ data "ibm_database_backups" "backup_database" {
 }
 
 # New postgresql instance pointing to the backup instance
-module "restored_postgresql_db" {
+module "restored_icd_postgresql" {
   source             = "../.."
   resource_group_id  = module.resource_group.resource_group_id
   name               = "${var.prefix}-postgres-restored"
   pg_version         = var.pg_version
   region             = var.region
-  resource_tags      = var.resource_tags
+  tags               = var.resource_tags
   access_tags        = var.access_tags
   member_host_flavor = "multitenant"
   backup_crn         = var.postgresql_db_backup_crn == null ? data.ibm_database_backups.backup_database[0].backups[0].backup_id : var.postgresql_db_backup_crn

examples/backup/outputs.tf

@@ -1,17 +1,12 @@
 ##############################################################################
 # Outputs
 ##############################################################################
-output "id" {
-  description = "Postgresql instance id"
-  value       = var.postgresql_db_backup_crn == null ? module.postgresql_db[0].id : null
-}
-
-output "restored_postgresql_db_id" {
+output "restored_icd_postgresql_id" {
   description = "Restored Postgresql instance id"
-  value       = module.restored_postgresql_db.id
+  value       = module.restored_icd_postgresql.id
 }
 
-output "restored_postgresql_db_version" {
+output "restored_icd_postgresql_version" {
   description = "Restored Postgresql instance version"
-  value       = module.restored_postgresql_db.version
+  value       = module.restored_icd_postgresql.version
 }

examples/basic/main.tf

@@ -11,18 +11,25 @@ module "resource_group" {
 }
 
 ##############################################################################
-# ICD postgresql database
+# Postgresql
 ##############################################################################
 
-module "postgresql_db" {
+module "database" {
   source             = "../.."
   resource_group_id  = module.resource_group.resource_group_id
-  name               = "${var.prefix}-postgres"
+  name               = "${var.prefix}-data-store"
   pg_version         = var.pg_version
   region             = var.region
-  resource_tags      = var.resource_tags
+  tags               = var.resource_tags
   access_tags        = var.access_tags
+  service_endpoints  = var.service_endpoints
   member_host_flavor = var.member_host_flavor
+  service_credential_names = {
+    "postgresql_admin" : "Administrator",
+    "postgresql_operator" : "Operator",
+    "postgresql_viewer" : "Viewer",
+    "postgresql_editor" : "Editor",
+  }
 }
 
 # On destroy, we are seeing that even though the replica has been returned as
@@ -34,7 +41,7 @@ module "postgresql_db" {
 # adding a time sleep here.
 
 resource "time_sleep" "wait_time" {
-  depends_on = [module.postgresql_db]
+  depends_on = [module.database]
 
   destroy_duration = "5m"
 }
@@ -49,10 +56,10 @@ module "read_only_replica_postgresql_db" {
   resource_group_id  = module.resource_group.resource_group_id
   name               = "${var.prefix}-read-only-replica-${count.index}"
   region             = var.region
-  resource_tags      = var.resource_tags
+  tags               = var.resource_tags
   access_tags        = var.access_tags
   pg_version         = var.pg_version
-  remote_leader_crn  = module.postgresql_db.crn
+  remote_leader_crn  = module.database.crn
   member_host_flavor = "multitenant"
   member_memory_mb   = 4096 # Must be an increment of 384 megabytes. The minimum size of a read-only replica is 2 GB RAM, new hosting model minimum is 4 GB RAM.
   member_disk_mb     = 5120 # Must be an increment of 512 megabytes. The minimum size of a read-only replica is 5 GB of disk

examples/basic/outputs.tf

@@ -3,31 +3,36 @@
 ##############################################################################
 output "id" {
   description = "Postgresql instance id"
-  value       = module.postgresql_db.id
+  value       = module.database.id
 }
 
 output "version" {
   description = "Postgresql instance version"
-  value       = module.postgresql_db.version
+  value       = module.database.version
 }
 
 output "adminuser" {
   description = "Database admin user name"
-  value       = module.postgresql_db.adminuser
+  value       = module.database.adminuser
 }
 
 output "hostname" {
   description = "Database connection hostname"
-  value       = module.postgresql_db.hostname
+  value       = module.database.hostname
 }
 
 output "port" {
   description = "Database connection port"
-  value       = module.postgresql_db.port
+  value       = module.database.port
 }
 
 output "certificate_base64" {
   description = "Database connection certificate"
-  value       = module.postgresql_db.certificate_base64
+  value       = module.database.certificate_base64
   sensitive   = true
 }
+
+output "postgresql_crn" {
+  description = "Postgresql CRN"
+  value       = module.database.crn
+}

examples/basic/variables.tf

@@ -56,3 +56,14 @@ variable "member_host_flavor" {
   description = "Allocated host flavor per member. For more information, see https://registry.terraform.io/providers/IBM-Cloud/ibm/latest/docs/resources/database#host_flavor"
   default     = null
 }
+
+variable "service_endpoints" {
+  type        = string
+  description = "The type of endpoint of the database instance. Possible values: `public`, `private`, `public-and-private`."
+  default     = "public"
+
+  validation {
+    condition     = can(regex("public|public-and-private|private", var.service_endpoints))
+    error_message = "Valid values for service_endpoints are 'public', 'public-and-private', and 'private'"
+  }
+}

examples/complete/main.tf

@@ -30,7 +30,7 @@ module "key_protect_all_inclusive" {
   resource_tags             = var.resource_tags
   keys = [
     {
-      key_ring_name = "icd-pg"
+      key_ring_name = "icd"
       keys = [
         {
           key_name     = local.data_key_name
@@ -101,7 +101,7 @@ module "cbr_zone" {
 # Postgres Instance
 ##############################################################################
 
-module "postgresql_db" {
+module "icd_postgresql" {
   source            = "../../"
   resource_group_id = module.resource_group.resource_group_id
   name              = "${var.prefix}-postgres"
@@ -112,9 +112,9 @@ module "postgresql_db" {
   # Example of how to use different KMS keys for data and backups
   use_ibm_owned_encryption_key = false
   use_same_kms_key_for_backups = false
-  kms_key_crn                  = module.key_protect_all_inclusive.keys["icd-pg.${local.data_key_name}"].crn
-  backup_encryption_key_crn    = module.key_protect_all_inclusive.keys["icd-pg.${local.backups_key_name}"].crn
-  resource_tags                = var.resource_tags
+  kms_key_crn                  = module.key_protect_all_inclusive.keys["icd.${local.data_key_name}"].crn
+  backup_encryption_key_crn    = module.key_protect_all_inclusive.keys["icd.${local.backups_key_name}"].crn
+  tags                         = var.resource_tags
   service_credential_names = {
     "postgressql_admin" : "Administrator",
     "postgressql_operator" : "Operator",
@@ -164,7 +164,7 @@ module "postgresql_db" {
 
 # VPE provisioning should wait for the database provisioning
 resource "time_sleep" "wait_120_seconds" {
-  depends_on      = [module.postgresql_db]
+  depends_on      = [module.icd_postgresql]
   create_duration = "120s"
 }
 
@@ -179,7 +179,7 @@ module "vpe" {
   cloud_service_by_crn = [
     {
       service_name = "${var.prefix}-postgres"
-      crn          = module.postgresql_db.crn
+      crn          = module.icd_postgresql.crn
     },
   ]
   vpc_id             = module.vpc.vpc_id

examples/complete/outputs.tf

@@ -3,42 +3,42 @@
 ##############################################################################
 output "id" {
   description = "Postgresql instance id"
-  value       = module.postgresql_db.id
+  value       = module.icd_postgresql.id
 }
 
 output "guid" {
   description = "Postgresql instance guid"
-  value       = module.postgresql_db.guid
+  value       = module.icd_postgresql.guid
 }
 
 output "version" {
   description = "Postgresql instance version"
-  value       = module.postgresql_db.version
+  value       = module.icd_postgresql.version
 }
 
 output "service_credentials_json" {
   description = "Service credentials json map"
-  value       = module.postgresql_db.service_credentials_json
+  value       = module.icd_postgresql.service_credentials_json
   sensitive   = true
 }
 
 output "service_credentials_object" {
   description = "Service credentials object"
-  value       = module.postgresql_db.service_credentials_object
+  value       = module.icd_postgresql.service_credentials_object
   sensitive   = true
 }
 
 output "cbr_rule_ids" {
   description = "CBR rule ids created to restrict Postgresql"
-  value       = module.postgresql_db.cbr_rule_ids
+  value       = module.icd_postgresql.cbr_rule_ids
 }
 
 output "hostname" {
   description = "Postgresql instance hostname"
-  value       = module.postgresql_db.hostname
+  value       = module.icd_postgresql.hostname
 }
 
 output "port" {
   description = "Postgresql instance port"
-  value       = module.postgresql_db.port
+  value       = module.icd_postgresql.port
 }