Skip to content

[ENH] Use k8s manifests/kustomize to deploy dask-gateway #2991

@marcelovilla

Description

@marcelovilla

#2609 was recently merged and it includes the logic to deploy k8s manifests and Helm charts directly instead of doing it via Terraform. I think we should build on top of that approach to deploy the dask-gateway Helm chart.

Originally posted by @marcelovilla in #2620

The goal for this issue is to refactor this stage:
https://github.com/nebari-dev/nebari/tree/main/src/_nebari/stages/kubernetes_services/template/modules/kubernetes/services/dask-gateway
and move all the configurations that we are doing right now through terraform resources, and instead use the available dask-gateway helm chart

How we would deploy this service will also change; it will have its own service and must follow a very similar structure as https://github.com/nebari-dev/nebari/tree/main/src/_nebari/stages/kubernetes_kuberhealthy

For whoever picks this issue, I would start familiarizing myself with how the kuberhealthy stage works, and then make a copy of it for the dask-gateway service, and start by replacing the associated yaml values based on what we are doing here

resource "kubernetes_deployment" "gateway" {
metadata {
name = "${var.name}-daskgateway-gateway"
namespace = var.namespace
}
spec {
replicas = 1
selector {
match_labels = {
"app.kubernetes.io/component" = "dask-gateway-gateway"
}
}
template {
metadata {
labels = {
"app.kubernetes.io/component" = "dask-gateway-gateway"
}
annotations = {
# This lets us autorestart when the secret changes!
"checksum/config-map" = sha256(jsonencode(kubernetes_config_map.gateway.data))
"checksum/secret" = sha256(jsonencode(kubernetes_secret.gateway.data))
}
}
spec {
affinity {
node_affinity {
required_during_scheduling_ignored_during_execution {
node_selector_term {
match_expressions {
key = var.general-node-group.key
operator = "In"
values = [var.general-node-group.value]
}
}
}
}
}

Note that a few resources, like PVC/PV/Secrets and others, might already be handled by the helm chart, so you will need to address the gaps accordingly.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status

    New 🚦

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions