Verify that all nodes actually joined - Debian

[fredrikscode]

Expected Behavior

I run ansible-playbook site.yml -i inventory/k3s-cluster/hosts.ini and get a k3s-cluster deployed.

Current Behavior

Runs through the playbook until the verification process which it goes through without success.

Steps to Reproduce

1. Modify hosts.ini with ip's
2. Add variable to all.yml: ansible_ssh_private_key_file: ~/.ssh/ansible
3. Run site.yml using: ansible-playbook site.yml -i inventory/k3s-cluster/hosts.ini

Context (variables)

Operating system: Debian 11

Hardware: (CPU/RAM/Disk type)

• Intel i5-8250U, 8GB, SSD
• Intel i3-9100, 64GB, SSD
• AMD Ryzen 5 PRO 1500, 24GB, SSD

Variables Used:

all.yml

k3s_version: v1.24.3+k3s1
ansible_user: ansible
ansible_ssh_private_key_file: ~/.ssh/ansible
systemd_dir: /etc/systemd/system
system_timezone: "Europe/Stockholm"
flannel_iface: "eth0"
apiserver_endpoint: "10.0.0.100"
k3s_token: "strawberrycake"
extra_server_args: "--disable servicelb --disable traefik"
extra_agent_args: ""
kube_vip_tag_version: "v0.5.0"
metal_lb_speaker_tag_version: "v0.13.4"
metal_lb_controller_tag_version: "v0.13.4"
metal_lb_ip_range: "10.0.0.80-10.0.0.90"

Hosts

host.ini

[master]
k3s-master-01
k3s-master-02
k3s-master-03

[node]
k3s-worker-01
k3s-worker-02
k3s-worker-03

[k3s_cluster:children]
master
node

Possible Solution

[timothystewart6]

This seems to be a Debian specific issue. It works fine with Ubuntu

➜  kubectl get nodes
NAME     STATUS   ROLES                       AGE    VERSION
k3s-01   Ready    control-plane,etcd,master   113s   v1.24.3+k3s1
k3s-02   Ready    control-plane,etcd,master   95s    v1.24.3+k3s1
k3s-03   Ready    control-plane,etcd,master   73s    v1.24.3+k3s1
k3s-04   Ready    <none>                      28s    v1.24.3+k3s1
k3s-05   Ready    <none>                      29s    v1.24.3+k3s1

[fredrikscode]

Hm, weird. I'll try with Ubuntu in a minute or two

[fredrikscode]

Tested with no more modifications other than what has been documented above and I still experience the same thing using Ubuntu 22.10. Also tested with 20.04 so I can't seem to blame this on that unfortunately.

Ended up with this error, which I've seen on Debian as well:

fatal: [k3s-master-01]: FAILED! => {"attempts": 20, "changed": false, "cmd": ["k3s", "kubectl", "get", "nodes", "-l", "node-role.kubernetes.io/master=true", "-o=jsonpath={.items[*].metadata.name}"], "delta": "0:00:00.104545", "end": "2022-08-20 22:50:14.638833", "msg": "", "rc": 0, "start": "2022-08-20 22:50:14.534288", "stderr": "", "stderr_lines": [], "stdout": "ubuntu", "stdout_lines": ["ubuntu"]}

[timothystewart6]

Not sure, I can run it multiple times in my lab (just did at least 20x to fix a bug). Are you sure you're using the correct ethernet adapter name?

[timothystewart6]

Check this to be sure #20

[ChrisThePCGeek]

I was working with OP in discord. the vm clones may all have the same hostname in /etc/hostname causing duplicate node names preventing them from joining into the cluster.

[fredrikscode]

I'm trying to confirm the issue now. Updating here shortly.

[fredrikscode]

This has been resolved and the issue was that my masters/nodes didn't have a proper hostname set.