Use external module for EFS

Author: vivgoyal-aws

.github/workflows/tfsec.yml

@@ -9,7 +9,7 @@ on:
   push:
     branches: [ "main" ]
   pull_request:
-    branches: [ "main" ]  
+    branches: [ "main" ]
   schedule:
     - cron: '39 18 * * 2'
 
@@ -29,10 +29,10 @@ jobs:
       - name: Run tfsec
         uses: aquasecurity/tfsec-sarif-action@9a83b5c3524f825c020e356335855741fd02745f
         with:
-          sarif_file: tfsec.sarif         
+          sarif_file: tfsec.sarif
 
       - name: Upload SARIF file
         uses: github/codeql-action/upload-sarif@v2
         with:
           # Path to SARIF file relative to the root of the repository
-          sarif_file: tfsec.sarif  
+          sarif_file: tfsec.sarif

README-PORTABLE.md

@@ -29,7 +29,9 @@ The solution has following features and benefits:
 
 - The target AWS Account and AWS Region are identified.
 - The AWS User/Role executing the Terraform scripts must have permissions to provision the target resources.
-- The Terraform CLI (`version = ">= 1.0.4"`) is installed.
+- The [Terraform CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli?in=terraform/aws-get-started) (`version = ">= 1.1.9"`) is installed.
+- The [Python 3.9+](https://www.python.org/downloads/) is installed.
+- AWS SDK for Python [boto3 1.24+](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#installation) is installed.
 - Terraform backend provider and state locking providers are identified and bootstrapped.
   - An [example bootstrap](https://github.com/aws-samples/aws-tf-transfer-sftp-efs/tree/main/bootstrap) module/example is provided that provisions an Amazon S3 bucket for Terraform state storage and Amazon DynamoDB table for Terraform state locking.
     - The Amazon S3 bucket name has to be globally unique.
@@ -49,7 +51,10 @@ The solution has following features and benefits:
 ## Usage
 
 - Use the module via [GitHub source](https://www.terraform.io/language/modules/sources#github) or copy the module into your repository.
-- Incorporate the module in your CI/CD pipeline as appropriate.
+- Incorporate the module in your infrastructure/storage [CI](https://aws.amazon.com/devops/continuous-integration/)/[CD](https://aws.amazon.com/devops/continuous-delivery/) [pipeline](https://docs.aws.amazon.com/codepipeline/latest/userguide/concepts.html) as appropriate.
+- This solution uses following external modules
+  - [aws-tf-kms](https://github.com/aws-samples/aws-tf-kms) to provision AWS KMS Key, if encryption is enabled and `kms_alias` is not provided.
+  - [aws-tf-efs](https://github.com/aws-samples/aws-tf-efs) to provision Amazon EFS or EFS Access Point, if `efs_id` is null or `efs_ap_id` is null.
 
 ## Scenarios
 

README.md

@@ -29,7 +29,9 @@ The solution has following features and benefits:
 
 - The target AWS Account and AWS Region are identified.
 - The AWS User/Role executing the Terraform scripts must have permissions to provision the target resources.
-- The Terraform CLI (`version = ">= 1.0.4"`) is installed.
+- The [Terraform CLI](https://learn.hashicorp.com/tutorials/terraform/install-cli?in=terraform/aws-get-started) (`version = ">= 1.1.9"`) is installed.
+- The [Python 3.9+](https://www.python.org/downloads/) is installed.
+- AWS SDK for Python [boto3 1.24+](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html#installation) is installed.
 - Terraform backend provider and state locking providers are identified and bootstrapped.
   - An [example bootstrap](./bootstrap) module/example is provided that provisions an Amazon S3 bucket for Terraform state storage and Amazon DynamoDB table for Terraform state locking.
     - The Amazon S3 bucket name has to be globally unique.
@@ -49,7 +51,10 @@ The solution has following features and benefits:
 ## Usage
 
 - Use the module via [GitHub source](https://www.terraform.io/language/modules/sources#github) or copy the module into your repository.
-- Incorporate the module in your CI/CD pipeline as appropriate.
+- Incorporate the module in your infrastructure/storage [CI](https://aws.amazon.com/devops/continuous-integration/)/[CD](https://aws.amazon.com/devops/continuous-delivery/) [pipeline](https://docs.aws.amazon.com/codepipeline/latest/userguide/concepts.html) as appropriate.
+- This solution uses following external modules
+  - [aws-tf-kms](https://github.com/aws-samples/aws-tf-kms) to provision AWS KMS Key, if encryption is enabled and `kms_alias` is not provided.
+  - [aws-tf-efs](https://github.com/aws-samples/aws-tf-efs) to provision Amazon EFS or EFS Access Point, if `efs_id` is null or `efs_ap_id` is null.
 
 ## Scenarios
 

examples/efs/scenario1/.terraform.lock.hcl

@@ -50,7 +50,7 @@ No providers.
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_common_efs"></a> [common\_efs](#module\_common\_efs) | ../../../modules/aws/efs | n/a |
+| <a name="module_common_efs"></a> [common\_efs](#module\_common\_efs) | github.com/aws-samples/aws-tf-efs//modules/aws/efs | v1.0.0 |
 
 ## Resources
 
@@ -66,7 +66,7 @@ No resources.
 | <a name="input_subnet_tags"></a> [subnet\_tags](#input\_subnet\_tags) | Tags to discover target subnets in the VPC, these tags should identify one or more subnets | `map(string)` | n/a | yes |
 | <a name="input_tags"></a> [tags](#input\_tags) | Common and mandatory tags for the resources | `map(string)` | n/a | yes |
 | <a name="input_vpc_tags"></a> [vpc\_tags](#input\_vpc\_tags) | Tags to discover target VPC, these tags should uniquely identify a VPC | `map(string)` | n/a | yes |
-| <a name="input_efs_access_point_specs"></a> [efs\_access\_point\_specs](#input\_efs\_access\_point\_specs) | List of EFS Access Point Specs to be created. It can be empty list. | <pre>list(object({<br>    efs_name        = string # unique name e.g. common<br>    efs_ap          = string # unique name e.g. common_sftp<br>    uid             = number<br>    gid             = number<br>    root_path       = string # e.g. /{env}/{project}/{purpose}/{name}<br>    owner_uid       = number # e.g. 0<br>    owner_gid       = number # e.g. 0<br>    root_permission = string # e.g. 0755<br>  }))</pre> | `[]` | no |
+| <a name="input_efs_access_point_specs"></a> [efs\_access\_point\_specs](#input\_efs\_access\_point\_specs) | List of EFS Access Point Specs to be created. It can be empty list. | <pre>list(object({<br>    efs_name        = string # unique name e.g. common<br>    efs_ap          = string # unique name e.g. common_sftp<br>    uid             = number<br>    gid             = number<br>    secondary_gids  = list(number)<br>    root_path       = string # e.g. /{env}/{project}/{purpose}/{name}<br>    owner_uid       = number # e.g. 0<br>    owner_gid       = number # e.g. 0<br>    root_permission = string # e.g. 0755<br>    principal_arns  = list(string)<br>  }))</pre> | `[]` | no |
 | <a name="input_efs_id"></a> [efs\_id](#input\_efs\_id) | EFS File System Id, if not provided a new EFS will be created | `string` | `null` | no |
 | <a name="input_kms_alias"></a> [kms\_alias](#input\_kms\_alias) | KMS Alias to discover KMS for EFS encryption, if not provided a new CMK will be created | `string` | `""` | no |
 | <a name="input_security_group_tags"></a> [security\_group\_tags](#input\_security\_group\_tags) | Tags used to discover EFS Security Group, if not provided new EFS security group will be created | `map(string)` | `null` | no |

examples/efs/scenario1/README.md

@@ -1,5 +1,5 @@
 module "common_efs" {
-  source = "../../../modules/aws/efs"
+  source = "github.com/aws-samples/aws-tf-efs//modules/aws/efs?ref=v1.0.0"
 
   region = var.region
 
@@ -14,18 +14,18 @@ module "common_efs" {
   kms_alias       = var.kms_alias
   kms_admin_roles = ["Admin"]
 
-  efs_specs = [
-    {
-      name             = "common"
-      efs_id           = var.efs_id
-      encrypted        = true
-      performance_mode = "generalPurpose"
-      transition_to_ia = "AFTER_7_DAYS"
-      backup_plan      = "EVERY-DAY"
-      # If security_group_tags is null, EFS security group is created
-      security_group_tags = var.security_group_tags
-    }
-  ]
+  # If security_group_tags is null, EFS security group is created
+  security_group_tags = var.security_group_tags
+
+  efs_name         = "common"
+  efs_id           = var.efs_id
+  encrypted        = true
+  performance_mode = "generalPurpose"
+  transition_to_ia = "AFTER_7_DAYS"
+
+  efs_tags = {
+    "BackupPlan" = "EVERY-DAY"
+  }
 
   efs_access_point_specs = var.efs_access_point_specs
 }

examples/efs/scenario1/main.tf

@@ -10,5 +10,5 @@ output "efs_ap" {
 
 output "efs_kms" {
   description = "KMS Keys created for EFS"
-  value       = module.common_efs.efs_kms
+  value       = module.common_efs.efs_kms_aliases
 }

examples/efs/scenario1/outputs.tf

@@ -6,11 +6,11 @@ region = "us-east-1"
 /*---------------------------------------------------------
 Common Variables
 ---------------------------------------------------------*/
-project  = "scenario1-efs"
+project  = "scenario1-efs-sftp"
 env_name = "dev"
 tags = {
   Env     = "DEV"
-  Project = "scenario1-efs"
+  Project = "scenario1-efs-sftp"
 }
 
 /*---------------------------------------------------------
@@ -40,9 +40,11 @@ efs_access_point_specs = [
     efs_ap          = "sftp_scenario1"
     uid             = 0
     gid             = 0
-    root_path       = "/dev/scenario1-efs/sftp/common"
+    secondary_gids  = []
+    root_path       = "/dev/scenario1-efs-sftp/sftp/common"
     owner_uid       = 0
     owner_gid       = 0
     root_permission = "0755"
+    principal_arns  = ["*"]
   }
 ]

examples/efs/scenario1/terraform.tfvars

@@ -65,10 +65,12 @@ variable "efs_access_point_specs" {
     efs_ap          = string # unique name e.g. common_sftp
     uid             = number
     gid             = number
+    secondary_gids  = list(number)
     root_path       = string # e.g. /{env}/{project}/{purpose}/{name}
     owner_uid       = number # e.g. 0
     owner_gid       = number # e.g. 0
     root_permission = string # e.g. 0755
+    principal_arns  = list(string)
   }))
   default = []
 }

examples/efs/scenario1/variables.tf

@@ -50,7 +50,7 @@ No providers.
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_common_efs"></a> [common\_efs](#module\_common\_efs) | ../../../modules/aws/efs | n/a |
+| <a name="module_common_efs"></a> [common\_efs](#module\_common\_efs) | github.com/aws-samples/aws-tf-efs//modules/aws/efs | v1.0.0 |
 
 ## Resources
 
@@ -66,7 +66,7 @@ No resources.
 | <a name="input_subnet_tags"></a> [subnet\_tags](#input\_subnet\_tags) | Tags to discover target subnets in the VPC, these tags should identify one or more subnets | `map(string)` | n/a | yes |
 | <a name="input_tags"></a> [tags](#input\_tags) | Common and mandatory tags for the resources | `map(string)` | n/a | yes |
 | <a name="input_vpc_tags"></a> [vpc\_tags](#input\_vpc\_tags) | Tags to discover target VPC, these tags should uniquely identify a VPC | `map(string)` | n/a | yes |
-| <a name="input_efs_access_point_specs"></a> [efs\_access\_point\_specs](#input\_efs\_access\_point\_specs) | List of EFS Access Point Specs to be created. It can be empty list. | <pre>list(object({<br>    efs_name        = string # unique name e.g. common<br>    efs_ap          = string # unique name e.g. common_sftp<br>    uid             = number<br>    gid             = number<br>    root_path       = string # e.g. /{env}/{project}/{purpose}/{name}<br>    owner_uid       = number # e.g. 0<br>    owner_gid       = number # e.g. 0<br>    root_permission = string # e.g. 0755<br>  }))</pre> | `[]` | no |
+| <a name="input_efs_access_point_specs"></a> [efs\_access\_point\_specs](#input\_efs\_access\_point\_specs) | List of EFS Access Point Specs to be created. It can be empty list. | <pre>list(object({<br>    efs_name        = string # unique name e.g. common<br>    efs_ap          = string # unique name e.g. common_sftp<br>    uid             = number<br>    gid             = number<br>    secondary_gids  = list(number)<br>    root_path       = string # e.g. /{env}/{project}/{purpose}/{name}<br>    owner_uid       = number # e.g. 0<br>    owner_gid       = number # e.g. 0<br>    root_permission = string # e.g. 0755<br>    principal_arns  = list(string)<br>  }))</pre> | `[]` | no |
 | <a name="input_efs_id"></a> [efs\_id](#input\_efs\_id) | EFS File System Id, if not provided a new EFS will be created | `string` | `null` | no |
 | <a name="input_kms_alias"></a> [kms\_alias](#input\_kms\_alias) | KMS Alias to discover KMS for EFS encryption, if not provided a new CMK will be created | `string` | `""` | no |
 | <a name="input_security_group_tags"></a> [security\_group\_tags](#input\_security\_group\_tags) | Tags used to discover EFS Security Group, if not provided new EFS security group will be created | `map(string)` | `null` | no |

examples/efs/scenario2/.terraform.lock.hcl

@@ -1,5 +1,5 @@
 module "common_efs" {
-  source = "../../../modules/aws/efs"
+  source = "github.com/aws-samples/aws-tf-efs//modules/aws/efs?ref=v1.0.0"
 
   region = var.region
 
@@ -14,18 +14,18 @@ module "common_efs" {
   kms_alias       = var.kms_alias
   kms_admin_roles = ["Admin"]
 
-  efs_specs = [
-    {
-      name             = "common"
-      efs_id           = var.efs_id
-      encrypted        = true
-      performance_mode = "generalPurpose"
-      transition_to_ia = "AFTER_7_DAYS"
-      backup_plan      = "EVERY-DAY"
-      # If security_group_tags is null, EFS security group is created
-      security_group_tags = var.security_group_tags
-    }
-  ]
+  # If security_group_tags is null, EFS security group is created
+  security_group_tags = var.security_group_tags
+
+  efs_name         = "common"
+  efs_id           = var.efs_id
+  encrypted        = true
+  performance_mode = "generalPurpose"
+  transition_to_ia = "AFTER_7_DAYS"
+
+  efs_tags = {
+    "BackupPlan" = "EVERY-DAY"
+  }
 
   efs_access_point_specs = var.efs_access_point_specs
 }

examples/efs/scenario2/README.md

@@ -10,5 +10,5 @@ output "efs_ap" {
 
 output "efs_kms" {
   description = "KMS Keys created for EFS"
-  value       = module.common_efs.efs_kms
+  value       = module.common_efs.efs_kms_aliases
 }

examples/efs/scenario2/main.tf

@@ -6,11 +6,11 @@ region = "us-east-1"
 /*---------------------------------------------------------
 Common Variables
 ---------------------------------------------------------*/
-project  = "scenario2-efs"
+project  = "scenario2-efs-sftp"
 env_name = "dev"
 tags = {
   Env     = "DEV"
-  Project = "scenario2-efs"
+  Project = "scenario2-efs-sftp"
 }
 
 /*---------------------------------------------------------

examples/efs/scenario2/outputs.tf

@@ -65,10 +65,12 @@ variable "efs_access_point_specs" {
     efs_ap          = string # unique name e.g. common_sftp
     uid             = number
     gid             = number
+    secondary_gids  = list(number)
     root_path       = string # e.g. /{env}/{project}/{purpose}/{name}
     owner_uid       = number # e.g. 0
     owner_gid       = number # e.g. 0
     root_permission = string # e.g. 0755
+    principal_arns  = list(string)
   }))
   default = []
 }

examples/efs/scenario2/terraform.tfvars

@@ -52,12 +52,12 @@ efs_id = "your-efs-id"
 efs_ap_id = "your-efs-ap-id"
 # Use existing EFS SG
 efs_sg_tags = {
-  Name = "scenario1-efs-common-efs-sg"
+  Name = "scenario1-efs-sftp-common-efs-sg"
   Env  = "DEV"
 }
 
 #efs exists, so kms must exist
-efs_kms_alias = "alias/scenario1-efs/efs"
+efs_kms_alias = "alias/scenario1-efs-sftp/efs"
 
 #create new roles
 user_role    = null