WARN[0157] [hostagent] Failed to run /usr/lib/qemu/virtiofsd --version (is this QEMU virtiofsd?): exit s tatus 1: setgroups() failed with error=1:Operation not permitted

[lattice0]

On a ubuntu 23.04 host on a intel i7 11th gen with KVM support, I get, when running:

limactl start \
  --name=$MACHINE_NAME \
  --cpus=$CPU_COUNT \
  --memory=$MEMORY \
  --tty=false \
  --mount-type=virtiofs \
  template://ubuntu

this error:

WARN[0157] [hostagent] Failed to run /usr/lib/qemu/virtiofsd --version (is this QEMU virtiofsd?): exit s
tatus 1: setgroups() failed with error=1:Operation not permitted
FATA[0157] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: s
ee "/home/lz/.lima/dev1/ha.stderr.log")

file output:

lz@lz:~$ cat /home/lz/.lima/dev1/ha.stderr.log
{"level":"debug","msg":"Creating iso file /home/lz/.lima/dev1/cidata.iso","time":"2023-09-26T23:0
7:42-03:00"}
{"level":"debug","msg":"Using /tmp/diskfs_iso1253774483 as workspace","time":"2023-09-26T23:07:42
-03:00"}
{"level":"debug","msg":"OpenSSH version 9.0.1 detected","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and
aes256-gcm@openssh.com","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"Start udp DNS listening on: 127.0.0.1:36003","time":"2023-09-26T23:07:42-
03:00"}
{"level":"debug","msg":"Start tcp DNS listening on: 127.0.0.1:32857","time":"2023-09-26T23:07:42-
03:00"}
{"level":"debug","msg":"QEMU version 7.2.0 detected","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"firmware candidates = [/home/lz/.local/share/qemu/edk2-x86_64-code.fd /us
r/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bi
n /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"Checking vhost directory /home/lz/.local/share/qemu/vhost-user","time":"2
023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"Failed to list vhost directory: open /home/lz/.local/share/qemu/vhost-use
r: no such file or directory","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"Checking vhost directory /usr/share/qemu/vhost-user","time":"2023-09-26T2
3:07:42-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-gpu.json","time":"2023-09-26T23:07:42-03:00
"}
{"level":"debug","msg":"{gpu /usr/lib/qemu/vhost-user-gpu}","time":"2023-09-26T23:07:42-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-virtiofsd.json","time":"2023-09-26T23:07:42
-03:00"}
{"level":"debug","msg":"{fs /usr/lib/qemu/virtiofsd}","time":"2023-09-26T23:07:42-03:00"}
{"level":"warning","msg":"Failed to run /usr/lib/qemu/virtiofsd --version (is this QEMU virtiofsd
?): exit status 1: setgroups() failed with error=1:Operation not permitted\n","time":"2023-09-26T
23:07:42-03:00"}
{"level":"fatal","msg":"Failed to locate virtiofsd","time":"2023-09-26T23:07:42-03:00"}

[afbjorklund]

You need to use the new Rust virtiofsd, not the "old" C virtiofsd

$ /usr/lib/qemu/virtiofsd --version
setgroups() failed with error=1:Operation not permitted

https://lima-vm.io/docs/config/mount/#virtiofs

• https://qemu.readthedocs.io/en/v7.2.0/tools/virtiofsd.html
  /usr/lib/qemu/virtiofsd
  virtiofsd version 7.2.0 (Debian 1:7.2+dfsg-5ubuntu2.3)

• https://gitlab.com/virtio-fs/virtiofsd
  /usr/local/bin/virtiofsd
  virtiofsd 1.8.0-dirty

[lattice0]

~/lima/bin$ virtiofsd --version
virtiofsd 1.8.0-dirty

and then

./limactl start --name virt  --mount-type=virtiofs   template://ubuntu
? Creating an instance "virt" Proceed with the current configuration
WARN[0001] `mountType: virtiofs` on Linux is experimental
INFO[0001] Attempting to download the image              arch=x86_64 digest="sha256:5ad255d32a30a2cda9f0df19f0a6ce8d6f3c81b63
845086a4cb5c43cf97fcb92" location="https://cloud-images.ubuntu.com/releases/23.04/release-20230810/ubuntu-23.04-server-cloudi
mg-amd64.img"
INFO[0002] Using cache "/home/lz/.cache/lima/download/by-url-sha256/e54a5ae9983387df7e867747dfd1d4b75b09df51588512504b67579b7
b7b4f1f/data"
INFO[0002] Attempting to download the nerdctl archive    arch=x86_64 digest="sha256:3f8c494e3c6a265fe2a3c41ef9d6bc859eeeb2209
5b3353d3558d8120833a23a" location="https://github.com/containerd/nerdctl/releases/download/v1.5.0/nerdctl-full-1.5.0-linux-am
d64.tar.gz"
INFO[0002] Using cache "/home/lz/.cache/lima/download/by-url-sha256/72a70b017cb55385b4c43d2bcc2a0333094089f1da0739a34c51c647b
f249674/data"
WARN[0002] [hostagent] Failed to run /usr/lib/qemu/virtiofsd --version (is this QEMU virtiofsd?): exit status 1: setgroups()
failed with error=1:Operation not permitted
FATA[0002] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/home/lz/.lima/vi
rt/ha.stderr.log")

error log:

~/lima/bin$ cat /home/lz/.lima/virt/ha.stderr.log
{"level":"debug","msg":"Creating iso file /home/lz/.lima/virt/cidata.iso","time":"2023-10-09T00:05:09-03:00"}
{"level":"debug","msg":"Using /tmp/diskfs_iso2044734303 as workspace","time":"2023-10-09T00:05:09-03:00"}
{"level":"debug","msg":"OpenSSH version 8.9.1 detected","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","tim
e":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Start udp DNS listening on: 127.0.0.1:33483","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Start tcp DNS listening on: 127.0.0.1:44987","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"QEMU version 6.2.0 detected","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"firmware candidates = [/home/lz/.local/share/qemu/edk2-x86_64-code.fd /usr/share/qemu/edk2-x86_64-cod
e.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-1
0-09T00:05:10-03:00"}
{"level":"debug","msg":"Checking vhost directory /home/lz/.local/share/qemu/vhost-user","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Failed to list vhost directory: open /home/lz/.local/share/qemu/vhost-user: no such file or directory
","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Checking vhost directory /usr/share/qemu/vhost-user","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-gpu.json","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"{gpu /usr/lib/qemu/vhost-user-gpu}","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-virtiofsd.json","time":"2023-10-09T00:05:10-03:00"}
{"level":"debug","msg":"{fs /usr/lib/qemu/virtiofsd}","time":"2023-10-09T00:05:10-03:00"}
{"level":"warning","msg":"Failed to run /usr/lib/qemu/virtiofsd --version (is this QEMU virtiofsd?): exit status 1: setgroups
() failed with error=1:Operation not permitted\n","time":"2023-10-09T00:05:10-03:00"}
{"level":"fatal","msg":"Failed to locate virtiofsd","time":"2023-10-09T00:05:10-03:00"}

[lattice0]

I moved virtiofsd to /usr/lib/qemu/ and then got

 cat /home/lz/.lima/virt/ha.stderr.log
{"level":"debug","msg":"Creating iso file /home/lz/.lima/virt/cidata.iso","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Using /tmp/diskfs_iso1865973165 as workspace","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"OpenSSH version 8.9.1 detected","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","tim
e":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Start udp DNS listening on: 127.0.0.1:55832","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Start tcp DNS listening on: 127.0.0.1:33517","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"QEMU version 6.2.0 detected","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"firmware candidates = [/home/lz/.local/share/qemu/edk2-x86_64-code.fd /usr/share/qemu/edk2-x86_64-cod
e.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-1
0-09T00:09:18-03:00"}
{"level":"debug","msg":"Checking vhost directory /home/lz/.local/share/qemu/vhost-user","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Failed to list vhost directory: open /home/lz/.local/share/qemu/vhost-user: no such file or directory
","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Checking vhost directory /usr/share/qemu/vhost-user","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-gpu.json","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"{fs /usr/lib/qemu/virtiofsd}","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"vhostCmd[0].Args: [/usr/lib/qemu/virtiofsd --socket-path /home/lz/.lima/virt/virtiofsd-0.sock --share
d-dir /home/lz]","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Try waiting for /home/lz/.lima/virt/virtiofsd-0.sock to appear (attempt 0)","time":"2023-10-09T00:09:
18-03:00"}
{"level":"debug","msg":"vhostCmd[1].Args: [/usr/lib/qemu/virtiofsd --socket-path /home/lz/.lima/virt/virtiofsd-1.sock --share
d-dir /tmp/lima]","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"virtiofsd-0[stderr]: thread 'main' panicked at src/sandbox.rs:408:21:","time":"2023-10-09T00:09:18-03
:00"}
{"level":"debug","msg":"virtiofsd-0[stderr]: couldn't setup id mappings: write to uid map failed: failed to execute newuidmap
: No such file or directory (os error 2)","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"virtiofsd-1[stderr]: thread 'main' panicked at src/sandbox.rs:408:21:","time":"2023-10-09T00:09:18-03
:00"}
{"level":"debug","msg":"virtiofsd-1[stderr]: couldn't setup id mappings: write to uid map failed: failed to execute newuidmap
: No such file or directory (os error 2)","time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"virtiofsd-0[stderr]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace","
time":"2023-10-09T00:09:18-03:00"}
{"level":"debug","msg":"Try waiting for /home/lz/.lima/virt/virtiofsd-1.sock to appear (attempt 0)","time":"2023-10-09T00:09:
18-03:00"}
{"level":"debug","msg":"virtiofsd-1[stderr]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace","
time":"2023-10-09T00:09:18-03:00"}
{"level":"error","msg":"Error from virtiofsd instance #0: exit status 1","time":"2023-10-09T00:09:18-03:00"}
{"level":"fatal","msg":"virtiofsd never created vhost socket: exit status 1","time":"2023-10-09T00:09:18-03:00"}

[lattice0]

Ok I installed ˋsudo apt install -y uidmapˋ and now I have

cat /home/lz/.lima/virt/ha.stderr.log
{"level":"debug","msg":"Creating iso file /home/lz/.lima/virt/cidata.iso","time":"2023-10-09T06:40:03-03:00"}
{"level":"debug","msg":"Using /tmp/diskfs_iso163240857 as workspace","time":"2023-10-09T06:40:03-03:00"}
{"level":"debug","msg":"OpenSSH version 8.9.1 detected","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"Start tcp DNS listening on: 127.0.0.1:35321","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"Start udp DNS listening on: 127.0.0.1:47929","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"QEMU version 6.2.0 detected","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"firmware candidates = [/home/lz/.local/share/qemu/edk2-x86_64-code.fd /usr/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd }{"level":"debug","msg":"Checking vhost directory /home/lz/.local/share/qemu/vhost-user","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"Failed to list vhost directory: open /home/lz/.local/share/qemu/vhost-user: no such file or directory","time":"2023-10-09T06:40:04-03:0}{"level":"debug","msg":"Checking vhost directory /usr/share/qemu/vhost-user","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-gpu.json","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"{gpu /usr/lib/qemu/vhost-user-gpu}","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"Checking vhost config 50-qemu-virtiofsd.json","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"{fs /usr/lib/qemu/virtiofsd}","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"vhostCmd[0].Args: [/usr/lib/qemu/virtiofsd --socket-path /home/lz/.lima/virt/virtiofsd-0.sock --shared-dir /home/lz]","time":"2023-10-0}{"level":"debug","msg":"Try waiting for /home/lz/.lima/virt/virtiofsd-0.sock to appear (attempt 0)","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"vhostCmd[1].Args: [/usr/lib/qemu/virtiofsd --socket-path /home/lz/.lima/virt/virtiofsd-1.sock --shared-dir /tmp/lima]","time":"2023-10-}{"level":"debug","msg":"Try waiting for /home/lz/.lima/virt/virtiofsd-1.sock to appear (attempt 0)","time":"2023-10-09T06:40:04-03:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/home/lz/.lima/virt/serial*.log\")","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"qCmd.Args: [/usr/bin/qemu-system-x86_64 -m 4096 -object memory-backend-file,id=virtiofs-shm,size=4294967296,mem-path=/dev/shm,share=on }{"level":"debug","msg":"virtiofsd-1[stderr]: [2023-10-09T09:40:04Z WARN  virtiofsd::sandbox] Couldn't set the process uid as root: -1","time":"2023-10-09T06:40}{"level":"debug","msg":"virtiofsd-1[stderr]: [2023-10-09T09:40:04Z WARN  virtiofsd::sandbox] Couldn't set the process gid as root: -1","time":"2023-10-09T06:40}{"level":"info","msg":"Waiting for the essential requirement 1 of 3: \"ssh\"","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"virtiofsd-0[stderr]: [2023-10-09T09:40:04Z WARN  virtiofsd::sandbox] Couldn't set the process uid as root: -1","time":"2023-10-09T06:40}{"level":"debug","msg":"virtiofsd-0[stderr]: [2023-10-09T09:40:04Z WARN  virtiofsd::sandbox] Couldn't set the process gid as root: -1","time":"2023-10-09T06:40}{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/home/lz/.lima/_config/user\" -o StrictHostKeyChecki}{"level":"debug","msg":"virtiofsd-0[stderr]: [2023-10-09T09:40:04Z INFO  virtiofsd] Waiting for vhost-user socket connection...","time":"2023-10-09T06:40:04-03}{"level":"debug","msg":"virtiofsd-1[stderr]: [2023-10-09T09:40:04Z INFO  virtiofsd] Waiting for vhost-user socket connection...","time":"2023-10-09T06:40:04-03}{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 34803: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdo}{"level":"debug","msg":"qemu[stderr]: Could not access KVM kernel module: Permission denied","time":"2023-10-09T06:40:04-03:00"}
{"level":"info","msg":"Driver stopped due to error: \"exit status 1\"","time":"2023-10-09T06:40:04-03:00"}
{"level":"info","msg":"Shutting down the host agent","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"shutting down the SSH master","time":"2023-10-09T06:40:04-03:00"}
{"level":"debug","msg":"executing ssh for exiting the master: /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/home/lz/.lima/_config/user\" -o StrictHostKeyCh}{"error":"failed to execute `ssh -O exit -p 34803 127.0.0.1`, out=\"Control socket connect(/home/lz/.lima/virt/ssh.sock): No such file or directory\\r\\n\": ex}{"level":"info","msg":"Shutting down QEMU with ACPI","time":"2023-10-09T06:40:04-03:00"}
{"error":"dial unix /home/lz/.lima/virt/qmp.sock: connect: connection refused","level":"warning","msg":"failed to open the QMP socket \"/home/lz/.lima/virt/qmp}{"level":"info","msg":"QEMU has already exited","time":"2023-10-09T06:40:04-03:00"}

[lattice0]

It appears that ˋsudo usermod -aG kvm $USER ˋ solved the problem