Skip to content

Commit e6d8530

Browse files
githubjianligithubjianli
authored
Merge pull request #143 from ExpediaGroup/feature/add_wd_memory_limit
feat: Added option to set guaranteed resource on waggledance pod
2 parents 3d67b00 + 5f60ef2 commit e6d8530

File tree

4 files changed

+30
-2
lines changed

4 files changed

+30
-2
lines changed

‎CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@ All notable changes to this project will be documented in this file.
33

44
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
55

6+
## [4.4.0] - 2024-04-18
7+
### Added
8+
- Add new variable `cpu_limit` and `memory_limit` to support set pod resource guaranteed.
9+
610
## [4.3.2] - 2024-03-26
711
### Updated
812
- Removed the datadog explicit provider configuration as suggested here: https://developer.hashicorp.com/terraform/language/modules/develop/providers#legacy-shared-modules-with-provider-configurations

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
1010
| aws_region | AWS region to use for resources. | string | - | yes |
1111
| bastion_ssh_key_secret_name | Secret name in AWS Secrets Manager which stores the private key used to log in to bastions. The secret's key should be `private_key` and the value should be stored as a base64 encoded string. Max character limit for a secret's value is 4096. | string | `` | no |
1212
| cpu | The number of CPU units to reserve for the Waggle Dance container. Valid values can be 256, 512, 1024, 2048 and 4096. Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `1024` | no |
13+
| cpu_limit | The number of CPU limit units to reserve for the Waggle Dance container. Valid values can be 256, 512, 1024, 2048 and 4096. It will use `cpu` * 1.25 if not specified. Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `1024` | no |
1314
| cpu_scale_in_cooldown | Cool down time(seconds) of scale in task by cpu usage | number | 300 | no |
1415
| cpu_scale_out_cooldown | Cool down time(seconds) of scale out task by cpu usage | number | 120 | no |
1516
| default_latency | Latency used for other (not primary) metastores that don't override it in their own configurations. See `latency` parameter in https://github.com/ExpediaGroup/waggle-dance/blob/main/README.md. | number | `0` | no |
@@ -32,6 +33,7 @@ For more information please refer to the main [Apiary](https://github.com/Expedi
3233
| k8s_max_replica_count | Max Number of k8s pod replicas to create. | number | `10` | no |
3334
| local_metastores | List of federated Metastore endpoints directly accessible on the local network. See section [`local_metastores`](#local_metastores) for more info. | list | `<list>` | no |
3435
| memory | The amount of memory (in MiB) used to allocate for the Waggle Dance container. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `4096` | no |
36+
| memory_limit | The amount of memory limit (in MiB) used to allocate for the Waggle Dance container, it will use `memory` * 1.25 if the limit is not specified. Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html | string | `null` | no |
3537
| primary_metastore_access_type | Primary Hive Metastore access control type. | string | `READ_AND_WRITE_ON_DATABASE_WHITELIST` | no |
3638
| primary_metastore_host | Primary Hive Metastore hostname configured in Waggle Dance. | string | `localhost` | no |
3739
| primary_metastore_port | Primary Hive Metastore port | string | `9083` | no |

‎k8s.tf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,11 @@
66

77
locals {
88
heapsize = ceil((var.memory * 85) / 100)
9-
memory_limit = ceil((var.memory * 120) / 100)
9+
memory_limit = length(var.memory_limit) != 0 ? var.memory_limit : ceil((var.memory * 120) / 100)
1010
actuator_port = 18000
1111
wd_port = 48869
1212
k8s_cpu = var.cpu / 1024
13-
k8s_cpu_limit = (var.cpu / 1024) * 1.25
13+
k8s_cpu_limit = length(var.cpu_limit) != 0 ? var.cpu_limit / 1024 : (var.cpu / 1024) * 1.25
1414
}
1515

1616
resource "kubernetes_service_account" "waggle_dance" {

‎variables.tf

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,17 @@ EOF
137137
default = "4096"
138138
}
139139

140+
variable "memory_limit" {
141+
description = <<EOF
142+
The amount of memory limit (in MiB) used to allocate for the Waggle Dance container. It will use memory * 1.25
143+
if this value is not specified.
144+
Valid values: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
145+
EOF
146+
147+
type = string
148+
default = ""
149+
}
150+
140151
variable "cpu" {
141152
description = <<EOF
142153
The number of CPU units to reserve for the Waggle Dance container.
@@ -148,6 +159,17 @@ EOF
148159
default = "1024"
149160
}
150161

162+
variable "cpu_limit" {
163+
description = <<EOF
164+
The number of CPU units limit to reserve for the Waggle Dance container.
165+
Valid values can be 256, 512, 1024, 2048 and 4096.It will use cpu * 1.25 if this value is not specified.
166+
Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html
167+
EOF
168+
169+
type = string
170+
default = ""
171+
}
172+
151173
variable "ingress_cidr" {
152174
description = "Generally allowed ingress CIDR list."
153175
type = list(string)

0 commit comments

Comments
 (0)