Skip to content
This repository was archived by the owner on Jul 16, 2024. It is now read-only.

Cloud Hypervisor Errors and Fixes

Jump to bottom
Dom edited this page Jul 5, 2023 · 13 revisions

virtualbox已经启动,此时再启动vm,此处提示设备忙

thread 'vmm' panicked at 'called `Result::unwrap()` on an `Err` value: VmCreate(Device or resource busy (os error 16))', vmm/src/vm.rs:863:41

启动vm里的--disk所指定的文件不存在时报错:

Error booting VM: VmBoot(DeviceManager(Disk(Os { code: 2, kind: NotFound, message: "No such file or directory" })))

VM netowrk,参考

Cloud-hypervisor-network-configuration

brctl stp br0 on,导致交换机的端口直接关闭,任何设备接入该端口都没有网络连接,慎用.

Proxy配置

[7] Couldn't connect to server (Failed to connect to 127.0.0.1 port 4781 after 0 ms: Connection refused); class=Net (12)

config.json 配置代理要用host的IP,而不是127.0.0.1

/root/.docker/config.json

 {
  "proxies":
  {
    "default":
    {
      "httpProxy": "socks5://10.0.0.41:4781",
      "httpsProxy": "socks5://10.0.0.41:4781",
      "noProxy": "*.test.example.com,.example2.com,127.0.0.0/8"
    }
  }
 }

重启网卡

ifconfig eno2 0

会把eno2关闭,下次再打开,输入ifconfig eno2 up但此时没有获取IP,需要再执行dhclient eno2

wget不支持socks5代理

wget --quiet

该命令除了不打印下载过程,连报错也不打印,把quiet删了后才打印出error:

Error parsing proxy URL socks5://10.0.0.41:4781: Unsupported scheme 'socks5'.

wget配置http代理

Error parsing proxy URL socks5://10.0.0.41:4781: Unsupported scheme 'socks5'.

把/root/.docker/config.json里的socks5://10.0.0.41:4781换成 http://10.0.0.41:4780 就好了

MSHV

cargo build --no-default-features --features kvm,mshv --all --release --target aarch64-unknown-linux-gnu
use of undeclared crate or module `mshv`

mshv是微软贡献 Linux 内核代码，可运行多个 Windows

mshv: 增加对检测嵌套的 hypervisor 的支持

hv, mshv : 改变嵌套 root 分区的中断向量

boot_time

Test 'boot_time_ms' running .. (control: test_timeout = 2s, test_iterations = 10, overrides: )

thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', test_infra/src/lib.rs:1232:18

See Performance-Metrics

tty config

启动vm命令:

sudo ./target/release/cloud-hypervisor \
          --kernel ../linux-cloud-hypervisor/arch/arm64/boot/Image  \
          --disk path=$ROOTFS --disk path=/tmp/ubuntu-cloudinit.img  \
          --cmdline "console=hvc0 root=/dev/vda1 rw" \
          --cpus boot=4   \
          --memory size=0,shared=on  \
          --memory-zone id=mem0,size=1G,shared=on,host_numa_node=0 \
          --net "tap=,mac=,ip=,mask=" \
          --serial tty \
          --console off

vm启动几秒就卡住

[    1.200267] EXT4-fs (vda1): re-mounted. Opts: (null). Quota mode: disabled.
[    1.888490] squashfs: SQUASHFS error: Xattrs in filesystem, these will be ignored
[    1.890036] unable to read xattr id index table
or
[    0.001761] Console: colour dummy device 80x25
[    0.002461] printk: console [tty0] enabled
[    0.003070] printk: bootconsole [pl11] disabled
cloud-hypervisor: 698.505152ms: <vcpu2> WARN:devices/src/legacy/uart_pl011.rs:358 -- [Debug I/O port: Kernel code: 0x41] 0.695055 seconds

此时VM仍在正常工作，只是console里不打印了。

通过ssh可以登录VM。

以下是几种不同配置的不同效果：

--kernel Image --cmdline "console=ttyAMA0" --serial tty --console off  kernel加载ttyAMA0的驱动，可以打印出完整的开机log和login prompt
--kernel Image --serial tty --console off                              未指定tty，只能通过ssh登录，在ssh中echo test > /dev/ttyAMA0，VM中显示test
--kernel Image --cmdline "console=hvc0"                                默认配置是--console tty,kernel 加载hvc0的驱动，该配置有开机log，有login prompt
--kernel Image                                                         不显示开机log，但显示login prompt，在ssh中echo test > /dev/hvc0，VM中显示test

--kernel CLOUDHV_EFI.fd --serial tty --console off                     此时--cmdline无效,使用disk里的/boot/grub/grub.cfg里的配置，有3秒kernel开机log，
                                                                       但没有ubuntu log，有Press ESCAPE for boot options，有login prompt，
                                                                       如果需要ubuntu log，可以手动在grub.cfg里增加console=ttyAMA0
--kernel CLOUDHV_EFI.fd --serial off --console off                     没有任何log，没有login prompt

如果不指定--console，其默认值为tty

OVS_DPDK

The following command blocks on ubuntu20.04

ovs-vsctl set Open_vSwitch . other_config:dpdk-init=true

Log in /var/log/openvswitch/ovs-vswitchd.log:

2023-04-23T02:20:26.062Z|00086|dpdk(ovs-vswitchd)|INFO|EAL ARGS: ovs-vswitchd --socket-mem 1024,1024,1024,1024,1024,1024,1024,1024 --socket-limit 1024,1024,1024,1024,1024,1024,1024,1024 -l 0.
2023-04-23T02:20:26.068Z|00089|dpdk(ovs-vswitchd)|ERR|EAL: invalid parameters for --socket-mem
2023-04-23T02:20:26.068Z|00090|dpdk(ovs-vswitchd)|ERR|EAL: Invalid 'command line' arguments.
2023-04-23T02:20:26.068Z|00091|dpdk(ovs-vswitchd)|EMER|Unable to initialize DPDK: Invalid argument

There are two solutions:

1.Modify ANC mode from Quadrant to Monolithic in BIOS of Ampera Altra,it reduces the number of NUMA.
2.run 'ovs-vsctl set Open_vSwitch' on ubuntu22.04,it has new version of openvswitch.
Clone this wiki locally