fix(patches): Ensure resulting NutanixMachineTemplate patches are clean #1236

thunderboltsid · 2025-07-23T19:28:37Z

These changes ensure the resulting NutanixMachineTemplate patches also reflect
the XOR between failureDomains and cluster/subnets.

How has this been tested?
Tested in conjunction with #1233 in #1226

Run make dev.run-on-kind and see default NutanixMachineTemplates when management cluster is created and observe no placeholder cluster and subnet fields

- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nutanix-quick-start-cp-nmt
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: ClusterClass
      name: nutanix-quick-start
  spec:
    template:
      spec:
        bootType: legacy
        image:
          name: ""
          type: name
        memorySize: 4Gi
        systemDiskSize: 40Gi
        vcpuSockets: 2
        vcpusPerSocket: 1
- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nutanix-quick-start-md-nmt
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: ClusterClass
      name: nutanix-quick-start
  spec:
    template:
      spec:
        bootType: legacy
        image:
          name: ""
          type: name
        memorySize: 4Gi
        systemDiskSize: 40Gi
        vcpuSockets: 2
        vcpusPerSocket: 1

Create a cluster without failure domains with a machine deployment with cluster and subnet set

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  ...
  name: nkp-sid-caren-no-fd
spec:
  ...
  topology:
    class: nutanix-quick-start
    controlPlane:
      metadata: {}
      replicas: 3
    variables:
    - name: clusterConfig
      value:
        ...
        controlPlane:
          nutanix:
            machineDetails:
              bootType: uefi
              imageLookup:
                baseOS: rocky-9.6
                format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
              memorySize: 4Gi
              systemDiskSize: 40Gi
              vcpuSockets: 2
              vcpusPerSocket: 1
              cluster:
                type: name
                name: ncn-dev-sandbox
              subnets:
              - type: name
                name: vlan173
        ...
    - name: workerConfig
      value:
        nutanix:
          machineDetails:
            bootType: uefi
            cluster:
              name: ncn-dev-sandbox
              type: name
            imageLookup:
              baseOS: rocky-9.6
              format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
            memorySize: 4Gi
            subnets:
            - name: vlan173
              type: name
            systemDiskSize: 40Gi
            vcpuSockets: 2
            vcpusPerSocket: 1
    version: 1.33.1
    workers:
      machineDeployments:
      - class: default-worker
        metadata:
          annotations:
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1"
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1"
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              nutanix:
                machineDetails:
                  bootType: uefi
                  imageLookup:
                    baseOS: rocky-9.6
                    format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
                  memorySize: 8Gi
                  cluster:
                    type: name
                    name: ncn-dev-sandbox
                  subnets:
                  - type: name
                    name: vlan173
                  systemDiskSize: 80Gi
                  vcpuSockets: 8
                  vcpusPerSocket: 1
      - class: default-worker
        metadata:
          annotations:
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1"
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1"
        name: md-1
        variables:
          overrides:
          - name: workerConfig
            value:
              nutanix:
                machineDetails:
                  bootType: uefi
                  imageLookup:
                    baseOS: rocky-9.6
                    format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
                  memorySize: 8Gi
                  cluster:
                    type: uuid
                    uuid: 00061f7f-44f7-19dc-3be1-7cc25586ee44
                  subnets:
                  - type: name
                    name: vlan173
                  systemDiskSize: 80Gi
                  vcpuSockets: 8
                  vcpusPerSocket: 1

Observe the generated NutanixMachineTemplate has cluster and subnets set correctly

apiVersion: v1
items:
- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nkp-sid-caren-no-fd-7np7m
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: nkp-sid-caren-no-fd
  spec:
    template:
      spec:
        bootType: uefi
        cluster:
          name: ncn-dev-sandbox
          type: name
        imageLookup:
          baseOS: rocky-9.6
          format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
        memorySize: 4Gi
        subnet:
        - name: vlan173
          type: name
        systemDiskSize: 40Gi
        vcpuSockets: 2
        vcpusPerSocket: 1
- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nkp-sid-caren-no-fd-md-0-qnr58
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: nkp-sid-caren-no-fd
  spec:
    template:
      spec:
        bootType: uefi
        cluster:
          name: ncn-dev-sandbox
          type: name
        imageLookup:
          baseOS: rocky-9.6
          format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
        memorySize: 8Gi
        subnet:
        - name: vlan173
          type: name
        systemDiskSize: 80Gi
        vcpuSockets: 8
        vcpusPerSocket: 1
- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nkp-sid-caren-no-fd-md-1-59tlq
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: nkp-sid-caren-no-fd
  spec:
    template:
      spec:
        bootType: uefi
        cluster:
          type: uuid
          uuid: 00061f7f-44f7-19dc-3be1-7cc25586ee44
        imageLookup:
          baseOS: rocky-9.6
          format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
        memorySize: 8Gi
        subnet:
        - name: vlan173
          type: name
        systemDiskSize: 80Gi
        vcpuSockets: 8
        vcpusPerSocket: 1

Create a cluster with control plane and machine deployment on failure domains

apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  labels:
    cluster.x-k8s.io/cluster-name: nkp-sid-caren
    cluster.x-k8s.io/provider: nutanix
  name: nkp-sid-caren
spec:
  ...
  topology:
    class: nutanix-quick-start
    controlPlane:
      metadata: {}
      replicas: 3
    variables:
    - name: clusterConfig
      value:
        ...
        controlPlane:
          nutanix:
            failureDomains:
            - fd-1
            - fd-2
            - fd-3
            machineDetails:
              bootType: uefi
              imageLookup:
                baseOS: rocky-9.6
                format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
              memorySize: 4Gi
              systemDiskSize: 40Gi
              vcpuSockets: 2
              vcpusPerSocket: 1
        ...
    - name: workerConfig
      value:
        nutanix:
          machineDetails:
            bootType: uefi
            cluster:
              name: ncn-dev-sandbox
              type: name
            imageLookup:
              baseOS: rocky-9.6
              format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
            memorySize: 4Gi
            subnets:
            - name: vlan173
              type: name
            systemDiskSize: 40Gi
            vcpuSockets: 2
            vcpusPerSocket: 1
    version: 1.33.1
    workers:
      machineDeployments:
      - class: default-worker
        failureDomain: fd-1
        metadata:
          annotations:
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1"
            cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1"
        name: md-0
        variables:
          overrides:
          - name: workerConfig
            value:
              nutanix:
                machineDetails:
                  bootType: uefi
                  imageLookup:
                    baseOS: rocky-9.6
                    format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
                  memorySize: 8Gi
                  systemDiskSize: 80Gi
                  vcpuSockets: 8
                  vcpusPerSocket: 1

Observe the generated NutanixMachineTemplate does not have cluster and subnet set

- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nkp-sid-caren-kp4nc
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: nkp-sid-caren
  spec:
    template:
      spec:
        bootType: uefi
        imageLookup:
          baseOS: rocky-9.6
          format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
        memorySize: 4Gi
        systemDiskSize: 40Gi
        vcpuSockets: 2
        vcpusPerSocket: 1
- apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
  kind: NutanixMachineTemplate
  metadata:
    ...
    name: nkp-sid-caren-md-0-bfm6v
    namespace: default
    ownerReferences:
    - apiVersion: cluster.x-k8s.io/v1beta1
      kind: Cluster
      name: nkp-sid-caren
  spec:
    template:
      spec:
        bootType: uefi
        imageLookup:
          baseOS: rocky-9.6
          format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-*
        memorySize: 8Gi
        systemDiskSize: 80Gi
        vcpuSockets: 8
        vcpusPerSocket: 1

remove subnet and cluster

we shouldn't be injecting extraneous values: this is confusing for debugging behaviour.

add new failure domain and meta mutation debug log.

Use XOR instead of OR for cluster and subnet presence check against failure domains on control plane and worker overrides. **How has this been tested?** Tested in conjunction with #1236 in #1226 * Create a cluster with control plane and machine deployments with failure domains with cluster and subnet variables also set in controlPlane machineDetails and machineDeployment variables overrides. ```yaml apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: labels: cluster.x-k8s.io/cluster-name: nkp-sid-caren cluster.x-k8s.io/provider: nutanix name: nkp-sid-caren spec: ... topology: class: nutanix-quick-start controlPlane: metadata: {} replicas: 3 variables: - name: clusterConfig value: ... controlPlane: nutanix: failureDomains: - fd-1 - fd-2 - fd-3 machineDetails: bootType: uefi cluster: type: name name: ncn-dev-sandbox subnets: - type: name name: vlan173 imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 4Gi systemDiskSize: 40Gi vcpuSockets: 2 vcpusPerSocket: 1 ... - name: workerConfig value: nutanix: machineDetails: bootType: uefi cluster: name: ncn-dev-sandbox type: name imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 4Gi subnets: - name: vlan173 type: name systemDiskSize: 40Gi vcpuSockets: 2 vcpusPerSocket: 1 version: 1.33.1 workers: machineDeployments: - class: default-worker failureDomain: fd-1 metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1" cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1" name: md-0 variables: overrides: - name: workerConfig value: nutanix: machineDetails: bootType: uefi cluster: type: name name: ncn-dev-sandbox subnets: - type: name name: vlan173 imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 8Gi systemDiskSize: 80Gi vcpuSockets: 8 vcpusPerSocket: 1 ``` and expect failures from the webhook ```sh $ k apply -f nkp-sid-caren-neg.yaml ... Error from server (Forbidden): error when creating "nkp-sid-caren-neg.yaml": admission webhook "cluster-validator.caren.nutanix.com" denied the request: [spec.topology.variables.clusterConfig.value.controlPlane.nutanix.machineDetails.cluster: Forbidden: "cluster" must not be set when failureDomains are configured., spec.topology.variables.clusterConfig.value.controlPlane.nutanix.machineDetails.subnets: Forbidden: "subnets" must not be set when failureDomains are configured., spec.topology.workers.machineDeployments.variables.overrides.workerConfig.value.nutanix.machineDetails.cluster: Forbidden: "cluster" must not be set when failureDomain is configured., spec.topology.workers.machineDeployments.variables.overrides.workerConfig.value.nutanix.machineDetails.subnets: Forbidden: "subnets" must not be set when failureDomain is configured.] ``` * Create a cluster without failure domains with a machine deployment with cluster and subnet not set ```yaml apiVersion: cluster.x-k8s.io/v1beta1 kind: Cluster metadata: labels: cluster.x-k8s.io/cluster-name: nkp-sid-caren-no-fd cluster.x-k8s.io/provider: nutanix name: nkp-sid-caren-no-fd spec: ... topology: class: nutanix-quick-start controlPlane: metadata: {} replicas: 3 variables: - name: clusterConfig value: ... controlPlane: nutanix: machineDetails: bootType: uefi imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 4Gi systemDiskSize: 40Gi vcpuSockets: 2 vcpusPerSocket: 1 ... - name: workerConfig value: nutanix: machineDetails: bootType: uefi cluster: name: ncn-dev-sandbox type: name imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 4Gi subnets: - name: vlan173 type: name systemDiskSize: 40Gi vcpuSockets: 2 vcpusPerSocket: 1 version: 1.33.1 workers: machineDeployments: - class: default-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1" cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1" name: md-0 variables: overrides: - name: workerConfig value: nutanix: machineDetails: bootType: uefi imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 8Gi cluster: type: name name: ncn-dev-sandbox subnets: - type: name name: vlan173 systemDiskSize: 80Gi vcpuSockets: 8 vcpusPerSocket: 1 - class: default-worker metadata: annotations: cluster.x-k8s.io/cluster-api-autoscaler-node-group-max-size: "1" cluster.x-k8s.io/cluster-api-autoscaler-node-group-min-size: "1" name: md-1 variables: overrides: - name: workerConfig value: nutanix: machineDetails: bootType: uefi imageLookup: baseOS: rocky-9.6 format: nkp-{{.BaseOS}}-release-{{.K8sVersion}}-* memorySize: 8Gi systemDiskSize: 80Gi vcpuSockets: 8 vcpusPerSocket: 1 ``` and expect failures from webhook ```sh $ k apply -f no-fd-neg.yaml ... Error from server (Forbidden): error when creating "no-fd-neg.yaml": admission webhook "cluster-validator.caren.nutanix.com" denied the request: [spec.topology.variables.clusterConfig.value.controlPlane.nutanix.machineDetails.cluster: Required value: "cluster" must be set when failureDomains are not configured., spec.topology.variables.clusterConfig.value.controlPlane.nutanix.machineDetails.subnets: Required value: "subnets" must be set when failureDomains are not configured., spec.topology.workers.machineDeployments.variables.overrides.workerConfig.value.nutanix.machineDetails.cluster: Required value: "cluster" must be set when failureDomain is not configured., spec.topology.workers.machineDeployments.variables.overrides.workerConfig.value.nutanix.machineDetails.subnets: Required value: "subnets" must be set when failureDomain is not configured.] ```

thunderboltsid added 5 commits July 23, 2025 21:23

fix(deps): Update CAPX to v1.7.0-beta.4

8bccae1

fix(clusterclass): update NutanixMachineTemplate in clusterclass

f81d3ea

remove subnet and cluster

fix(handler): remove extraneous injection for nutanix

92304dd

we shouldn't be injecting extraneous values: this is confusing for debugging behaviour.

fix(log): instrument better debug logging

6d0e4e6

add new failure domain and meta mutation debug log.

fix(test): update the patches for machinedetails test

8387305

github-actions bot added fix and removed fix labels Jul 23, 2025

This was referenced Jul 23, 2025

fix(webhook): Update cluster webhook to validate failure domains #1233

Merged

fix(webhook): Update cluster webhook to validate failure domains #1226

Closed

thunderboltsid requested review from jimmidyson, dlipovetsky, dkoshkin, supershal and yanhua121 July 23, 2025 19:32

dkoshkin approved these changes Jul 23, 2025

View reviewed changes

github-actions bot added fix and removed fix labels Jul 23, 2025

jimmidyson approved these changes Jul 24, 2025

View reviewed changes

jimmidyson merged commit 4bbfcf8 into jira/NCN-108599 Jul 24, 2025
33 checks passed

jimmidyson deleted the jira/NCN-108599-part2 branch July 24, 2025 09:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(patches): Ensure resulting NutanixMachineTemplate patches are clean #1236

fix(patches): Ensure resulting NutanixMachineTemplate patches are clean #1236

Uh oh!

thunderboltsid commented Jul 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

fix(patches): Ensure resulting NutanixMachineTemplate patches are clean #1236

fix(patches): Ensure resulting NutanixMachineTemplate patches are clean #1236

Uh oh!

Conversation

thunderboltsid commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

thunderboltsid commented Jul 23, 2025 •

edited

Loading