Skip to content

Commit e02bad8

Browse files
penguinjournalshayk99iru
authored
feat: cloudconnector deployment on AppRunner (#85)
* feat: apprunner poc * feat: doc update * chore: add name to apprunner service name to prevent name colision * feat: cloudconnector deploy on apprunner * fix: changed name for apprunner task to fix aws requirements of length * chore: linter fixing * doc: apprunner diagram updated * chore: sequential test * chore: sequential test * chore: sequential test * chorus: timeout increased to pass tests on CI * chorus: ci test region changed to fit apprunner * chore: name per test * docs: snippet fixed * chore: fix naming length * feat: retrieve cloud connector from new registry * docs: readme updated to explain the apprunner option * docs: updated broken image links * docs: fixed typo error * docs: several naming fixes * feat: versions upgraded * docs: updated url for diagram to fit master * docs: versions updated Co-authored-by: hayk99 <hayk.kocharyan@sysdig.com> Co-authored-by: iru <irune.prado@sysdig.com>
1 parent 3218a1b commit e02bad8

File tree

83 files changed

+849
-56
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

83 files changed

+849
-56
lines changed

.github/workflows/ci-integration-tests.yaml

Lines changed: 41 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ on:
1515
concurrency: terraform
1616

1717
jobs:
18-
integration_test-eks:
18+
integration_test_eks:
1919
concurrency: terraform-account
2020
continue-on-error: true
2121

@@ -99,7 +99,7 @@ jobs:
9999
run: bundle exec kitchen destroy "organizational-k8s-aws"
100100

101101
integration_test_ecs:
102-
needs: integration_test-eks
102+
needs: integration_test_eks
103103
concurrency: terraform-account
104104
continue-on-error: true
105105

@@ -118,20 +118,20 @@ jobs:
118118
ruby-version: 2.7
119119
bundler-cache: true
120120

121-
- name: Run single-account test
121+
- name: Run single-account-ecs test
122122
env:
123123
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
124124
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
125125
AWS_REGION: ${{ secrets.AWS_REGION }}
126-
run: bundle exec kitchen test "single-account-aws"
126+
run: bundle exec kitchen test "single-account-ecs-aws"
127127

128-
- name: Destroy single-account resources
128+
- name: Destroy single-account-ecs resources
129129
env:
130130
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
131131
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
132132
AWS_REGION: ${{ secrets.AWS_REGION }}
133133
if: ${{ failure() }}
134-
run: bundle exec kitchen destroy "single-account-aws"
134+
run: bundle exec kitchen destroy "single-account-ecs-aws"
135135

136136
- name: Run organizational test
137137
env:
@@ -149,3 +149,38 @@ jobs:
149149
TF_VAR_sysdig_secure_for_cloud_member_account_id: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCOUNT_ID }}
150150
if: ${{ failure() }}
151151
run: bundle exec kitchen destroy "organizational-aws"
152+
153+
integration_test_app_runner:
154+
needs: integration_test_ecs
155+
concurrency: terraform-account
156+
continue-on-error: true
157+
158+
name: Test-Kitchen-AppRunner
159+
runs-on: ubuntu-latest
160+
env:
161+
TF_VAR_sysdig_secure_url: https://secure.sysdig.com
162+
TF_VAR_sysdig_secure_api_token: ${{secrets.KUBELAB_SECURE_API_TOKEN}}
163+
164+
steps:
165+
- name: Checkout
166+
uses: actions/checkout@v2
167+
168+
- uses: ruby/setup-ruby@v1
169+
with:
170+
ruby-version: 2.7
171+
bundler-cache: true
172+
173+
- name: Run single-account-apprunner test
174+
env:
175+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
176+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
177+
AWS_REGION: "eu-west-1" #APPRUNNER ONLY RUNS ON SOME REGIONS https://docs.aws.amazon.com/general/latest/gr/apprunner.html
178+
run: bundle exec kitchen test "single-account-apprunner-aws"
179+
180+
- name: Destroy single-account-apprunner resources
181+
env:
182+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_QA_CLOUDNATIVE_ACCESS_KEY_ID }}
183+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_QA_CLOUDNATIVE_SECRET_ACCESS_KEY }}
184+
AWS_REGION: "eu-west-1" #APPRUNNER ONLY RUNS ON SOME REGIONS https://docs.aws.amazon.com/general/latest/gr/apprunner.html
185+
if: ${{ failure() }}
186+
run: bundle exec kitchen destroy "single-account-apprunner-aws"

.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,5 +42,5 @@ override.tf.json
4242

4343
# test
4444
.kitchen/*
45-
test/fixtures/single-account/.kitchen/
45+
test/fixtures/single-account-ecs/.kitchen/
4646
test/snippets/*

.kitchen.yml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ driver:
44
root_module_directory: test/fixtures
55
parallelism: 4
66
verify_version: false
7+
command_timeout: 1200
78

89
provisioner:
910
name: terraform
@@ -12,9 +13,9 @@ platforms:
1213
- name: "aws"
1314

1415
suites:
15-
- name: single-account
16+
- name: single-account-ecs
1617
driver:
17-
root_module_directory: test/fixtures/single-account
18+
root_module_directory: test/fixtures/single-account-ecs
1819
- name: single-account-k8s
1920
driver:
2021
root_module_directory: test/fixtures/single-account-k8s
@@ -24,3 +25,6 @@ suites:
2425
- name: organizational-k8s
2526
driver:
2627
root_module_directory: test/fixtures/organizational-k8s
28+
- name: single-account-apprunner
29+
driver:
30+
root_module_directory: test/fixtures/single-account-apprunner

CONTRIBUTE.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@
3232

3333
- [ ] **modules** (infra or services) have been modified?
3434
- [ ] a `README.md` file has been added to the folder
35-
- [ ] if modules are relevant to usage-case understanding `diagram.py/png` have been updated accodingly
35+
- [ ] if modules are relevant to usage-case understanding `diagram.py/png` have been updated accordingly. To re-generate diagrams yo need to run `python diagram.py` and need diagram installed `pip install diagrams`.
3636
- [ ] if pre-requirements have been modified, update accordingly on
3737
- [ ] README's
3838
- [ ] Sysdig docs

README.md

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -51,25 +51,37 @@ For other Cloud providers check: [GCP](https://github.com/sysdiglabs/terraform-g
5151
## Usage
5252

5353
- There are several ways to deploy this in you AWS infrastructure, gathered under **[`/examples`](./examples)**
54+
- [Single Account on ECS](#--single-account-on-ecs)
55+
- [Single Account on AppRunner](#--single-account-on-apprunner)
56+
- [Single-Account with a pre-existing Kubernetes Cluster](#--single-account-with-a-pre-existing-kubernetes-cluster)
57+
- [Organizational](#--organizational)
5458
- Many module,examples and use-cases provide ways to **re-use existing resources (as optionals)** in your infrastructure (cloudtrail, ecs, vpc, k8s cluster,...)
5559
- Find some real **use-case scenario explanations** under [`/examples-internal/use-cases*`](./examples-internal)
5660
- [Single Account - Existing Cloudtrail](./examples-internal/use-cases-reuse-resources/single-existing-cloudtrail.md)
5761
- [Organizational - Existing Cloudtrail, ECS, VPC, Subnet](./examples-internal/use-cases-reuse-resources/org-existing-cloudtrail-ecs-vpc-subnet.md)
5862
- [Organizational - Existing Cloudtrail withouth SNS, but with S3 configuration, with K8s Cluster and Filtered Cloudtrail Event Account](./examples-internal/use-cases-self-baked/org-s3-k8s-filtered-account.md)
5963

60-
### - Single-Account
64+
### - Single-Account on ECS
6165

6266
Sysdig workload will be deployed in the same account where user's resources will be watched.<br/>
63-
More info in [`./examples/single-account`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account)
67+
More info in [`./examples/single-account-ecs`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-ecs)
6468

65-
![single-account diagram](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account/diagram-single.png)
69+
![single-account diagram](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account-ecs/diagram-single.png)
6670

71+
### - Single-Account on AppRunner
72+
73+
Sysdig workload will be deployed using AppRunner in the same account where user's resources will be watched.<br/>
74+
More info in [`./examples/single-account-apprunner`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-apprunner)
75+
76+
![single-account diagram on apprunner](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account-apprunner/diagram-single.png)
6777

6878
### - Single-Account with a pre-existing Kubernetes Cluster
6979

7080
If you already own a Kubernetes Cluster on AWS, you can use it to deploy Sysdig Secure for Cloud, instead of default ECS cluster.<br/>
7181
More info in [`./examples/single-account-k8s`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/tree/master/examples/single-account-k8s)
7282

83+
![single-account with pre-existing kubernetes cluster](https://raw.githubusercontent.com/sysdiglabs/terraform-aws-secure-for-cloud/master/examples/single-account-k8s/diagram.png)
84+
7385
### - Organizational
7486

7587
Using an organizational configuration Cloudtrail.<br/>

examples-internal/use-cases-reuse-resources/single-existing-cloudtrail.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323

2424
## Suggested setup
2525

26-
For this use-case we're going to use the [`./examples/single-account`](../../examples/single-account/README.md) setup.
26+
For this use-case we're going to use the [`./examples/single-account-ecs`](../../examples/single-account-ecs/README.md) setup.
2727
In order for this setup to work, all resources must be in the same AWS account and region.
2828
Before proceeding, please read the example README and check whether you comply with requirements.
2929

@@ -86,7 +86,7 @@ provider "aws" {
8686
}
8787
8888
module "sysdig-s4c" {
89-
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account"
89+
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account-ecs"
9090
name = "sysdig-s4c"
9191
9292
cloudtrail_sns_arn = "<CLOUDRAIL_SNS_TOPIC_ARN>"

examples/organizational/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,7 @@ $ terraform apply
149149
| Name | Source | Version |
150150
|------|--------|---------|
151151
| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
152-
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
152+
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector-ecs | n/a |
153153
| <a name="module_cloudtrail"></a> [cloudtrail](#module\_cloudtrail) | ../../modules/infrastructure/cloudtrail | n/a |
154154
| <a name="module_codebuild"></a> [codebuild](#module\_codebuild) | ../../modules/infrastructure/codebuild | n/a |
155155
| <a name="module_ecs_vpc"></a> [ecs\_vpc](#module\_ecs\_vpc) | ../../modules/infrastructure/ecs-vpc | n/a |

examples/organizational/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ module "cloud_connector" {
5050
providers = {
5151
aws = aws.member
5252
}
53-
source = "../../modules/services/cloud-connector"
53+
source = "../../modules/services/cloud-connector-ecs"
5454
name = "${var.name}-cloudconnector"
5555

5656
secure_api_token_secret_name = module.ssm.secure_api_token_secret_name

examples/organizational/permissions.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ module "secure_for_cloud_role" {
1818
# secure_for_cloud_role <-> ecs_role trust relationship
1919
# note:
2020
# - definition of a ROOT lvl secure_for_cloud_connector_ecs_tas_role to avoid cyclic dependencies
21-
# - duplicated in ../../modules/services/cloud-connector/ecs-service-security.tf
21+
# - duplicated in ../../modules/services/cloud-connector-ecs/ecs-service-security.tf
2222
# -----------------------------------------------------------------
2323
resource "aws_iam_role" "connector_ecs_task" {
2424
provider = aws.member
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Sysdig Secure for Cloud in AWS<br/>[ Example :: App Runner ]
2+
3+
Deploy Sysdig Secure for Cloud in a single AWS account using App Runner.<br/>
4+
All the required resources and workloads will be run under the same account.
5+
6+
## Prerequisites
7+
8+
Minimum requirements:
9+
10+
1. Configure [Terraform **AWS** Provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
11+
1. Secure requirements, as input variable value
12+
```
13+
sysdig_secure_api_token=<SECURE_API_TOKEN>
14+
```
15+
16+
## Notice
17+
18+
* **Resource creation inventory** Find all the resources created by Sysdig examples in the resource-group `sysdig-secure-for-cloud` (AWS Resource Group & Tag Editor) <br/><br/>
19+
* **Deployment cost** This example will create resources that cost money.<br/>Run `terraform destroy` when you don't need them anymore <br/><br/>
20+
* **AppRunner enabled zones** AppRunner isn't available in all AWS zones, check [AppRunner Service endpoints](https://docs.aws.amazon.com/general/latest/gr/apprunner.html) for enabled zones.
21+
22+
23+
## Usage
24+
25+
For quick testing, use this snippet on your terraform files
26+
27+
```terraform
28+
terraform {
29+
required_providers {
30+
sysdig = {
31+
source = "sysdiglabs/sysdig"
32+
version = ">=0.5.33"
33+
}
34+
}
35+
}
36+
37+
provider "sysdig" {
38+
sysdig_secure_api_token = "<SYSDIG_SECURE_URL>"
39+
sysdig_secure_url = "<SYSDIG_SECURE_API_TOKEN"
40+
}
41+
42+
provider "aws" {
43+
region = "<AWS_REGION> Take care of AppRunner available zones: https://docs.aws.amazon.com/general/latest/gr/apprunner.html"
44+
}
45+
46+
module "cloudvision_aws_apprunner_single_account" {
47+
source = "sysdiglabs/secure-for-cloud/aws//examples/single-account-apprunner"
48+
name = "<APPRUNNER_SERVICE_NAME>"
49+
}
50+
```
51+
52+
See [inputs summary](#inputs) or module module [`variables.tf`](https://github.com/sysdiglabs/terraform-aws-secure-for-cloud/blob/master/examples/single-account/variables.tf) file for more optional configuration.
53+
54+
To run this example you need have your [aws account profile configured in CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) and to execute:
55+
```terraform
56+
$ terraform init
57+
$ terraform plan
58+
$ terraform apply
59+
```
60+
61+
<!-- BEGIN_TF_DOCS -->
62+
## Requirements
63+
64+
| Name | Version |
65+
|------|---------|
66+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.15.0 |
67+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.0.0 |
68+
| <a name="requirement_sysdig"></a> [sysdig](#requirement\_sysdig) | >= 0.5.33 |
69+
70+
## Providers
71+
72+
| Name | Version |
73+
|------|---------|
74+
| <a name="provider_sysdig"></a> [sysdig](#provider\_sysdig) | >= 0.5.33 |
75+
76+
## Modules
77+
78+
| Name | Source | Version |
79+
|------|--------|---------|
80+
| <a name="module_cloud_bench"></a> [cloud\_bench](#module\_cloud\_bench) | ../../modules/services/cloud-bench | n/a |
81+
| <a name="module_cloud_connector"></a> [cloud\_connector](#module\_cloud\_connector) | ../../modules/services/cloud-connector | n/a |
82+
| <a name="module_cloudtrail"></a> [cloudtrail](#module\_cloudtrail) | ../../modules/infrastructure/cloudtrail | n/a |
83+
| <a name="module_codebuild"></a> [codebuild](#module\_codebuild) | ../../modules/infrastructure/codebuild | n/a |
84+
| <a name="module_ecs_vpc"></a> [ecs\_vpc](#module\_ecs\_vpc) | ../../modules/infrastructure/ecs-vpc | n/a |
85+
| <a name="module_resource_group"></a> [resource\_group](#module\_resource\_group) | ../../modules/infrastructure/resource-group | n/a |
86+
| <a name="module_ssm"></a> [ssm](#module\_ssm) | ../../modules/infrastructure/ssm | n/a |
87+
88+
## Resources
89+
90+
| Name | Type |
91+
|------|------|
92+
| [sysdig_secure_connection.current](https://registry.terraform.io/providers/sysdiglabs/sysdig/latest/docs/data-sources/secure_connection) | data source |
93+
94+
## Inputs
95+
96+
| Name | Description | Type | Default | Required |
97+
|------|-------------|------|---------|:--------:|
98+
| <a name="input_benchmark_regions"></a> [benchmark\_regions](#input\_benchmark\_regions) | List of regions in which to run the benchmark. If empty, the task will contain all aws regions by default. | `list(string)` | `[]` | no |
99+
| <a name="input_cloudtrail_is_multi_region_trail"></a> [cloudtrail\_is\_multi\_region\_trail](#input\_cloudtrail\_is\_multi\_region\_trail) | true/false whether cloudtrail will ingest multiregional events | `bool` | `true` | no |
100+
| <a name="input_cloudtrail_kms_enable"></a> [cloudtrail\_kms\_enable](#input\_cloudtrail\_kms\_enable) | true/false whether cloudtrail delivered events to S3 should persist encrypted | `bool` | `true` | no |
101+
| <a name="input_cloudtrail_sns_arn"></a> [cloudtrail\_sns\_arn](#input\_cloudtrail\_sns\_arn) | ARN of a pre-existing cloudtrail\_sns. If defaulted, a new cloudtrail will be created | `string` | `"create"` | no |
102+
| <a name="input_deploy_benchmark"></a> [deploy\_benchmark](#input\_deploy\_benchmark) | Whether to deploy or not the cloud benchmarking | `bool` | `true` | no |
103+
| <a name="input_deploy_image_scanning_ecr"></a> [deploy\_image\_scanning\_ecr](#input\_deploy\_image\_scanning\_ecr) | true/false whether to deploy the image scanning on ECR pushed images | `bool` | `true` | no |
104+
| <a name="input_deploy_image_scanning_ecs"></a> [deploy\_image\_scanning\_ecs](#input\_deploy\_image\_scanning\_ecs) | true/false whether to deploy the image scanning on ECS running images | `bool` | `true` | no |
105+
| <a name="input_ecs_cluster_name"></a> [ecs\_cluster\_name](#input\_ecs\_cluster\_name) | Name of a pre-existing ECS (elastic container service) cluster. If defaulted, a new ECS cluster/VPC/Security Group will be created | `string` | `"create"` | no |
106+
| <a name="input_ecs_task_cpu"></a> [ecs\_task\_cpu](#input\_ecs\_task\_cpu) | Amount of CPU (in CPU units) to reserve for cloud-connector task | `string` | `"256"` | no |
107+
| <a name="input_ecs_task_memory"></a> [ecs\_task\_memory](#input\_ecs\_task\_memory) | Amount of memory (in megabytes) to reserve for cloud-connector task | `string` | `"512"` | no |
108+
| <a name="input_ecs_vpc_id"></a> [ecs\_vpc\_id](#input\_ecs\_vpc\_id) | ID of the VPC where the workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `string` | `"create"` | no |
109+
| <a name="input_ecs_vpc_region_azs"></a> [ecs\_vpc\_region\_azs](#input\_ecs\_vpc\_region\_azs) | List of Availability Zones for ECS VPC creation. e.g.: ["apne1-az1", "apne1-az2"]. If defaulted, two of the default 'aws\_availability\_zones' datasource will be taken | `list(string)` | `[]` | no |
110+
| <a name="input_ecs_vpc_subnets_private_ids"></a> [ecs\_vpc\_subnets\_private\_ids](#input\_ecs\_vpc\_subnets\_private\_ids) | List of VPC subnets where workload is to be deployed. Defaulted to be created when 'ecs\_cluster\_name' is not provided. | `list(string)` | `[]` | no |
111+
| <a name="input_name"></a> [name](#input\_name) | Name to be assigned to all child resources. A suffix may be added internally when required. Use default value unless you need to install multiple instances | `string` | `"sfc"` | no |
112+
| <a name="input_tags"></a> [tags](#input\_tags) | sysdig secure-for-cloud tags | `map(string)` | <pre>{<br> "product": "sysdig-secure-for-cloud"<br>}</pre> | no |
113+
114+
## Outputs
115+
116+
| Name | Description |
117+
|------|-------------|
118+
| <a name="output_cloudtrail_sns_topic_arn"></a> [cloudtrail\_sns\_topic\_arn](#output\_cloudtrail\_sns\_topic\_arn) | ARN of cloudtrail\_sns topic |
119+
<!-- END_TF_DOCS -->
120+
121+
122+
## Authors
123+
124+
Module is maintained and supported by [Sysdig](https://sysdig.com).
125+
126+
## License
127+
128+
Apache 2 Licensed. See LICENSE for full details.

0 commit comments

Comments
 (0)