How to deploy UserVolumeConfig?

[msundman78]

Sorry in advance for this newbe question. I'm deploying Talos 1.10.1 via Terraform Talos-provider 0.8.0 onto vCloud Director (VMware) and am trying to add a second disk as local storage during deployment.

I've added the disk as an independed disk in Terraform VCD and it deploys fine as I can see it as disk "nvme0n1" with "talosctl get disks".

I'm now trying to use UserVolumeConfig to create and mount a partition on this disk, but don't understand how to deploy it.

I've tried after deployment to apply it as YAML file but it fails with:

talosctl apply-config -f ./infrastructure/talos/user-volume-config.yaml       
error applying new configuration: 1 error occurred:
	* 100.100.16.30: rpc error: code = InvalidArgument desc = the applied machine configuration doesn't contain v1alpha1 config, did you mean to patch the machine config instead?

./infrastructure/talos/user-volume-config.yaml:

apiVersion: v1alpha1
kind: UserVolumeConfig
name: my-data 
provisioning:
    diskSelector:
        match: disk.transport == "nvme"
filesystem:
    type: xfs 

In my terraform I currently have:

data "talos_machine_configuration" "controller" {
  cluster_name     = var.k8s_cluster_name
  cluster_endpoint = var.k8s_cluster_endpoint
  machine_secrets  = talos_machine_secrets.talos.machine_secrets
  machine_type     = "controlplane"
  talos_version    = "1.10.1"
  kubernetes_version = var.k8s_version
  config_patches = [
    yamlencode({
      machine = {
        kubelet = {
          extraArgs = {
            rotate-server-certificates = true
          }
        }
        install = {
          disk = "/dev/sda"
        }

        network = {
          interfaces = [
            {
              interface = "eth0"
            }
          ]
        }
      }
      cluster = {
        allowSchedulingOnControlPlanes = true
        proxy = {
          disabled = false
        }
      }
    })
  ]
}

Is it possible to merge the UserVolumeConfig into "config_patches" above?

[msundman78]

I got it working by adding the following to my machine config:

        disks = [
          {
            device = "/dev/nvme0n1"
            partitions = [
              {
                mountpoint = "/var/mnt/data"
              }
            ]
          }
        ]

But as I understand it that way is deprecated, so I'm still struggling to understand how to get UserVolumeConfig working.

[msundman78]

After a lot of trial-n-error I got it working. I think I got confused about the old machine-config format and the new "K8s-style" config document format in combination with YAML vs HCL.

After converting YAML to HCL and embedded it as two separate yamlencode() blocks in the terraform configs I finally got it working:

data "talos_machine_configuration" "controller" {
  cluster_name     = var.k8s_cluster_name
  cluster_endpoint = var.k8s_cluster_endpoint
  machine_secrets  = talos_machine_secrets.talos.machine_secrets
  machine_type     = "controlplane"
  talos_version    = "1.10.1"
  kubernetes_version = var.k8s_version
  config_patches = [
    yamlencode({
      machine = {
        kubelet = {
          extraArgs = {
            rotate-server-certificates = true
          }
        }
        install = {
          disk = "/dev/sda"
        }
        network = {
          interfaces = [
            {
              interface = "eth0"
            }
          ]
        }
      }
      cluster = {
        allowSchedulingOnControlPlanes = true
        proxy = {
          disabled = false
        }
      }
    }),
    yamlencode({
      apiVersion = "v1alpha1"
      kind = "UserVolumeConfig"
      name = "data"
      provisioning = {
        diskSelector = {
          match = "disk.transport == \"nvme\""
        }
        minSize = "10GiB"
        grow = true
      }
      filesystem = {
        type = "xfs"
      } 
    })
  ]
}

After also adding "grow=true" it also auto-grows fine after resizing and rebooting.

Is there anyway to trigger a rescanning of SCSI/NVMEe disks and trigger a partition resizing online without rebooting the node?

[smira]

Is there anyway to trigger a rescanning of SCSI/NVMEe disks and trigger a partition resizing online without rebooting the node?

At the moment Talos does not support this without reboot.