Skip to content

Commit e9f2a8b

Browse files
author
Alexander Wiechert
committed
extend and modularize
1 parent dc203c9 commit e9f2a8b

20 files changed

+782
-298
lines changed

‎README.md

Lines changed: 4 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -1,114 +1,5 @@
1-
[![Terraform CI](https://github.com/elastic2ls-com/terraform-aws-finops-cur-grafana/actions/workflows/terraform.yml/badge.svg)](https://github.com/elastic2ls-com/terraform-aws-finops-cur-grafana/actions)
2-
![License](https://img.shields.io/badge/license-MIT-brightgreen?logo=mit)
3-
![Status](https://img.shields.io/badge/status-active-brightgreen.svg?logo=git)
4-
[![Sponsor](https://img.shields.io/badge/sponsors-AlexanderWiechert-blue.svg?logo=github-sponsors)](https://github.com/sponsors/AlexanderWiechert/)
5-
[![Contact](https://img.shields.io/badge/website-elastic2ls.com-blue.svg?logo=google-chrome)](https://www.elastic2ls.com/)
6-
[![Terraform Registry](https://img.shields.io/badge/download-blue.svg?logo=terraform&style=social)](https://registry.terraform.io/modules/elastic2ls-com/finops-cur-athena/aws/latest)
7-
![OpenTofu Compatible](https://img.shields.io/badge/OpenTofu-Compatible-4E9A06?logo=opentofu)
8-
9-
# terraform-aws-finops-cur-grafana
10-
11-
Terraform module to automate the setup of AWS Cost and Usage Reports (CUR) and make them queryable in Athena.
12-
13-
Supports:
14-
- CUR report generation (Parquet format)
15-
- Storage in S3 with proper permissions
16-
- Glue Crawler to create Athena table
17-
- Named Athena queries for FinOps reporting
18-
19-
This module is compatible with both Terraform (>=1.4) and OpenTofu (>=1.4).
20-
21-
---
22-
23-
## Features
24-
25-
- Provisions a secure S3 bucket for CUR reports
26-
- Enables daily Parquet-based AWS CUR reports with `RESOURCES` schema
27-
- Automatically registers CUR data via Glue Crawler
28-
- Makes CUR data queryable with Athena
29-
- Creates sample named Athena queries (e.g. EBS cost summary)
30-
- Works seamlessly with Grafana Athena integration
31-
- Example usage included
32-
33-
---
34-
35-
## Usage
36-
37-
### Example
38-
39-
```hcl
40-
module "cur_athena" {
41-
source = "github.com/elastic2ls-com/terraform-aws-finops-cur-grafana"
42-
43-
aws_region = "eu-central-1"
44-
s3_bucket_name = "my-cur-report-bucket"
45-
report_name = "ebs-cost-usage-report"
46-
report_prefix = "cur/ebs-report/"
47-
athena_database_name = "cur_database"
48-
report_table_name = "cur_table"
49-
crawler_name = "cur-crawler"
50-
}
511
```
52-
53-
Then run:
54-
55-
```bash
56-
terraform init
57-
terraform plan
58-
terraform apply
59-
```
60-
61-
---
62-
63-
## Variables
64-
65-
| Name | Description | Type | Default |
66-
|----------------------|---------------------------------------------|--------|------------------------|
67-
| aws_region | AWS region | string | "eu-central-1" |
68-
| s3_bucket_name | Name of the S3 bucket for CUR data | string | n/a (required) |
69-
| report_name | CUR report name | string | "ebs-cost-usage-report"|
70-
| report_prefix | Prefix in S3 for CUR output | string | "cur/ebs-report/" |
71-
| athena_database_name | Athena database name | string | "cur_database" |
72-
| report_table_name | Athena table name for CUR data | string | "cur_table" |
73-
| crawler_name | Name of the Glue Crawler | string | "cur-crawler" |
74-
75-
---
76-
77-
## Outputs
78-
79-
| Name | Description |
80-
|----------------------|-----------------------------------------|
81-
| s3_bucket_name | Name of the created S3 bucket |
82-
| athena_database_name | Name of the created Athena database |
83-
| cur_report_name | Name of the CUR report created |
84-
| named_query_id | ID of the predefined named Athena query |
85-
86-
---
87-
88-
## Requirements
89-
90-
- Terraform ≥ 1.4
91-
- AWS Provider ≥ 5.0
92-
93-
---
94-
95-
## CI/CD
96-
97-
This module includes GitHub Actions to perform:
98-
99-
- `terraform fmt`
100-
- `terraform validate`
101-
- `terraform plan` on examples
102-
- `checkov` security scan
103-
104-
---
105-
106-
## License
107-
108-
MIT
109-
110-
---
111-
112-
## Maintainers
113-
114-
[elastic2ls](https://github.com/elastic2ls-com)
2+
aws sso login --profile staffing-tool-test
3+
terraform init --backend-config=backend/test.tfbackend
4+
terraform plan -var-file=test.tfvars
5+
```

‎backends/test.tfbackend

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
region = "eu-central-1"
2+
bucket = "535159664206-terraform-state"
3+
dynamodb_table = "535159664206-terraform-state"
4+
key = "terraform.tfstate"
5+
encrypt = "true"
6+
profile = "staffing-tool-test"

‎data.tf

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
data "aws_caller_identity" "current" {}
2+
3+
data "aws_vpc" "ciss" {
4+
filter {
5+
name = "owner-id"
6+
values = ["086523049501"]
7+
}
8+
}
9+
10+
data "aws_subnets" "ciss_public" {
11+
filter {
12+
name = "vpc-id"
13+
values = [data.aws_vpc.ciss.id]
14+
}
15+
filter {
16+
name = "tag:Name"
17+
values = ["public"]
18+
}
19+
}
20+
21+
data "aws_subnet" "ciss_public" {
22+
for_each = toset(data.aws_subnets.ciss_public.ids)
23+
id = each.value
24+
}
25+
26+
data "aws_subnets" "ciss_internal" {
27+
filter {
28+
name = "vpc-id"
29+
values = [data.aws_vpc.ciss.id]
30+
}
31+
filter {
32+
name = "tag:Name"
33+
values = ["internal"]
34+
}
35+
}
36+
37+
data "aws_subnet" "ciss_internal" {
38+
for_each = toset(data.aws_subnets.ciss_internal.ids)
39+
id = each.value
40+
}
41+
42+
data "aws_subnets" "ciss_isolated" {
43+
filter {
44+
name = "vpc-id"
45+
values = [data.aws_vpc.ciss.id]
46+
}
47+
filter {
48+
name = "tag:Name"
49+
values = ["isolated"]
50+
}
51+
}
52+
53+
data "aws_subnet" "ciss_isolated" {
54+
for_each = toset(data.aws_subnets.ciss_isolated.ids)
55+
id = each.value
56+
}

‎examples/basic/main.tf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,8 @@ module "cur_athena" {
55
s3_bucket_name = "my-cur-report-bucket"
66
report_name = "ebs-cost-usage-report"
77
report_prefix = "cur/ebs-report/"
8-
athena_database_name = "cur_database"
8+
athena_database_name = "cur_catalog"
99
report_table_name = "cur_table"
1010
crawler_name = "cur-crawler"
11-
vpc_id = "vpc-1234567890abcdef0"
11+
aws_vpc = "vpc-1234567890abcdef0"
1212
}

‎main.tf

Lines changed: 29 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -1,109 +1,38 @@
11
module "grafana" {
22
source = "./modules/grafana"
3-
vpc_id = var.vpc_id
3+
aws_region = var.aws_region
4+
aws_vpc = var.aws_vpc
5+
subnet_ids = var.subnet_ids
46
project_name = var.project_name
5-
}
6-
7-
resource "aws_ecs_cluster" "this" {
8-
name = var.project_name
9-
}
10-
11-
# --- Load Balancer + Target Group ---
12-
resource "aws_lb" "grafana" {
13-
name = "${var.project_name}-alb"
14-
internal = false
15-
load_balancer_type = "application"
16-
security_groups = [aws_security_group.grafana_sg.id]
17-
subnets = var.subnet_ids
18-
}
19-
20-
resource "aws_lb_target_group" "grafana_tg" {
21-
name = "${var.project_name}-tg"
22-
port = 3000
23-
protocol = "HTTP"
24-
target_type = "ip"
25-
vpc_id = var.vpc_id
26-
27-
health_check {
28-
path = "/"
29-
port = "3000"
30-
protocol = "HTTP"
31-
matcher = "200-399"
32-
interval = 30
33-
timeout = 5
34-
healthy_threshold = 2
35-
unhealthy_threshold = 3
36-
}
37-
}
387

39-
resource "aws_lb_listener" "https" {
40-
load_balancer_arn = aws_lb.grafana.arn
41-
port = 3000
42-
protocol = "HTTP"
43-
44-
default_action {
45-
type = "forward"
46-
target_group_arn = aws_lb_target_group.grafana_tg.arn
47-
}
48-
}
49-
50-
# --- ECS Service ---
51-
resource "aws_ecs_service" "grafana" {
52-
name = "${var.project_name}-service"
53-
cluster = aws_ecs_cluster.this.id
54-
task_definition = module.grafana.task_definition_arn
55-
launch_type = "FARGATE"
56-
desired_count = 1
57-
58-
network_configuration {
59-
subnets = var.subnet_ids
60-
security_groups = [aws_security_group.grafana_sg.id]
61-
assign_public_ip = true
62-
}
63-
64-
load_balancer {
65-
target_group_arn = aws_lb_target_group.grafana_tg.arn
66-
container_name = "grafana"
67-
container_port = 3000
68-
}
8+
grafana_image = var.grafana_image
9+
grafana_admin_user = var.grafana_admin_user
10+
grafana_admin_password = var.grafana_admin_password
11+
grafana_ecs_network_mode = var.grafana_ecs_network_mode
12+
grafana_ecs_cpu = var.grafana_ecs_cpu
13+
grafana_ecs_memory = var.grafana_ecs_memory
14+
grafana_ecs_container_port = var.grafana_ecs_container_port
15+
grafana_ecs_log_group_name = var.grafana_ecs_log_group_name
16+
grafana_ecs_execution_role_name = var.grafana_ecs_execution_role_name
17+
18+
cur_catalog = module.cur.cur_catalog
19+
athena_result_location = module.cur.athena_result_location
20+
athena_workgroup = module.cur.athena_workgroup
6921
}
7022

71-
# --- DNS (optional) ---
72-
# resource "aws_route53_record" "grafana_dns" {
73-
# zone_id = var.zone_id
74-
# name = var.domain_name
75-
# type = "A"
76-
#
77-
# alias {
78-
# name = aws_lb.grafana.dns_name
79-
# zone_id = aws_lb.grafana.zone_id
80-
# evaluate_target_health = true
81-
# }
82-
# }
83-
84-
resource "aws_security_group" "grafana_sg" {
85-
name = "${var.project_name}-grafana-sg"
86-
description = "Security Group for Grafana"
87-
vpc_id = var.vpc_id
88-
89-
ingress {
90-
description = "Allow ALB to Grafana on port 3000"
91-
from_port = 3000
92-
to_port = 3000
93-
protocol = "tcp"
94-
cidr_blocks = ["0.0.0.0/0"] # Nur für Tests – später auf ALB Security Group einschränken!
95-
# security_groups = [var.alb_security_group_id] # Besser: explizite SG, siehe oben
96-
}
23+
module "cur" {
24+
source = "./modules/cur"
25+
aws_region = var.aws_region
26+
aws_vpc = var.aws_vpc
27+
subnet_ids = var.subnet_ids
28+
project_name = var.project_name
9729

98-
egress {
99-
description = "Allow all outbound traffic"
100-
from_port = 0
101-
to_port = 0
102-
protocol = "-1"
103-
cidr_blocks = ["0.0.0.0/0"]
104-
}
30+
cur_report_compression = var.cur_report_compression
31+
cur_report_format = var.cur_report_format
32+
cur_report_versioning = var.cur_report_versioning
10533

106-
tags = {
107-
Name = "${var.project_name}-grafana-sg"
108-
}
34+
cur_report_name = var.cur_report_name
35+
cur_s3_bucket_name = var.cur_s3_bucket_name
36+
cur_report_prefix = var.cur_report_prefix
37+
cur_report_time_unit = var.cur_report_time_unit
10938
}

0 commit comments

Comments
 (0)