This repository was archived by the owner on Jul 16, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 18
Virtual machine network configuration
Dom edited this page Aug 29, 2023
·
15 revisions
We can follow macvtap-bridge to configure network for VM.
# The MAC address must be attached to the macvtap and be used inside the guest
mac="c2:67:4f:53:29:cb"
# Host network adapter to bridge the guest onto
host_net="eno1"
# Create the macvtap0 as a new virtual MAC associated with the host network
sudo ip link add link "$host_net" name macvtap0 type macvtap
sudo ip link set macvtap0 address "$mac" up
sudo ip link show macvtap0
# A new character device is created for this interface
tapindex=$(< /sys/class/net/macvtap0/ifindex)
tapdevice="/dev/tap$tapindex"
# Ensure that we can access this device
sudo chown "$UID.$UID" "$tapdevice"
# Use --net fd=3 to point to fd 3 which the shell has opened to point to the /dev/tapN device
target/debug/cloud-hypervisor \
--kernel ~/src/linux/vmlinux \
--disk path=~/workloads/focal.raw \
--cpus boot=1 --memory size=512M \
--cmdline "root=/dev/vda1 console=hvc0" \
--net fd=3,mac=$mac 3<>$"$tapdevice"
To make '3<>/dev/tapX' work, don't run target/debug/cloud-hypervisor with sudo.
If the VM can't browse internet,please try dhclient in VM.
On host:
ifconfig
macvtap0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet6 fe80::c067:4fff:fe53:29cb prefixlen 64 scopeid 0x20<link>
ether c2:67:4f:53:29:cb txqueuelen 500 (Ethernet)
RX packets 30344 bytes 2616872 (2.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 126 bytes 11911 (11.9 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
+-----------------------------------------------------------------------------------------+
| |
| |
| |
| +---------------------------------+ |
| | | |
| | VM | |
| | | |
| | +--------------------+ | |
| | | enp0s4 | | |
| | | 10.10.0.99 | | |
| | | | | |
| | | c2:67:4f:53:29:cb | | |
| +-----+--------------------+------+ |
| | +------------------+ |
| | | | |
| | | macvtap0 | |
| | | no IP | |
| +----------------------- | | |
| |c2:67:4f:53:29:cb | |
| | | |
| +------------------+ |
| | +-------------------+
| Host | | enP9p3s0 |
| | | 10.10.0.41 |
| +-------------| 00:1b:21:e3:ec:50 |
| | |
+---------------------------------------------------------------------+-------------------+
Some more details:https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/5084.
We can follow the CH integration test as example to setup VM network using TAP backend. The integration test uses customized ubuntu cloudinit image. The image contains network configuration.
- echo -n "@DEFAULT_TCP_LISTENER_MESSAGE" > /dev/tcp/@HOST_IP/@TCP_LISTENER_PORT
+ echo -n "booted" > /dev/tcp/192.168.2.1/8001
The patch comes from test_infra/src/lib.rs:
prepare_cloudinit
user_data_string.replace("@DEFAULT_TCP_LISTENER_MESSAGE",DEFAULT_TCP_LISTENER_MESSAGE,);
user_data_string.replace("@TCP_LISTENER_PORT", &network.tcp_listener_port.to_string());
./scripts/create-cloud-init.sh
We can find id0 in test_data/cloud-init/ubuntu/network-config.
id0:
match:
macaddress: 12:34:56:78:90:ab
addresses:
- 192.168.2.2/24
gateway4: 192.168.2.1
There are also some other ids with different mac and ip addresses.
cp /root/workloads/focal-server-cloudimg-arm64-custom-20210929-0.raw /root/ch_files/osdisk.img
target/aarch64-unknown-linux-gnu/release/cloud-hypervisor \
--api-socket /tmp/ch0 \
--event-monitor path=/tmp/event.json \
--cpus boot=4 \
--memory size=4G,hotplug_method=virtio-mem,hotplug_size=32G \
--balloon size=0 \
--kernel /root/workloads/Image \
--disk path=/root/ch_files/osdisk.img \
--disk path=/tmp/ubuntu-cloudinit.img,iommu=on \
--net id=net123,tap=,mac=12:34:56:78:90:ab,ip=192.168.2.1,mask=255.255.255.0 \
--vsock cid=3,socket=/tmp/vsock \
--cmdline "console=hvc0 root=/dev/vda1 rw systemd.journald.forward_to_console=1"
Cloud-init detects the VM's mac address "12:34:56:78:90:ab" and sets the id0's ip address "192.168.2.2" to VM.
The relevant code is in test_infra/src/lib.rs:
GuestNetworkConfig
wait_vm_boot
let listener = TcpListener::bind(listen_addr.as_str()).map_err(WaitForBootError::Listen)?;
let num_events = match epoll::wait(epoll_fd, timeout * 1000_i32, &mut events[..])
listener.accept()
On host:
ifconfig
vmtap0: flags=67<UP,BROADCAST,RUNNING> mtu 1500
inet 192.168.2.1 netmask 255.255.255.0 broadcast 192.168.2.255
inet6 fe80::ec51:5ff:fe08:494f prefixlen 64 scopeid 0x20<link>
ether aa:16:ea:a7:03:93 txqueuelen 1000 (Ethernet)
RX packets 8 bytes 534 (534.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 16 bytes 1152 (1.1 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
On VM:
ip addr
enp0s4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 12:34:56:78:90:ab brd ff:ff:ff:ff:ff:ff
inet 192.168.2.2/24 brd 192.168.2.255 scope global enp0s4
valid_lft forever preferred_lft forever
+-------------------------------------------------------------------------------------------------------------------+
| +--------------------------------+ |
| | | |
| | VM | |
| | | |
| | +---------------------------+ | |
| | | enp0s4 192.168.2.2 | | |
| | | 12:34:56:78:90:ab | | |
| | | | | |
| +-+---------------------------+--+ |
| | |
| | |
| | |
| | |
| +---------------------------------+ |
| | | |
| | | |
| | vmtap0 | |
| | 192.168.2.1 | |
| | aa:16:ea:a7:03:93 | |
| | | |
| | | |
| +---------------------------------+ |
| |
| |
| Host |
| |
| |
| |
| |
| |
| |
+-------------------------------------------------------------------------------------------------------------------+
Some more details:https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/5274 and https://cloudinit.readthedocs.io/en/latest/
On VM:
sudo ip link set etp0s4 down
sudo ip addr del 192.168.2.2 dev enp0s4
On host:
brctl addbr virbr0
brctl addif virbr0 enP9p3s0 vmtap0
ifconfig
virbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:47:72:ce txqueuelen 1000 (Ethernet)
RX packets 78876 bytes 21617301 (21.6 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 910 bytes 50598 (50.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
On VM:
sudo dhclient
ip addr
2: enp0s4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 12:34:56:78:90:ab brd ff:ff:ff:ff:ff:ff
inet 10.10.0.121/22 brd 10.0.3.255 scope global dynamic enp0s4
valid_lft 7185sec preferred_lft 7185sec
+-------------------------------------------------------------------------------------------------------------------+
| +--------------------------------+ |
| | | |
| | VM | |
| | | |
| | +---------------------------+ | |
| | | enp0s4 10.10.0.121 | | |
| | | 12:34:56:78:90:ab | | |
| | | | | |
| +-+---------------------------+--+ |
| | |
| | |
| | |
| | |
| +---------------------------------+ +--------------------------------+ |
| | | | | |
| | | | | |
| | vmtap0 | ------+| |------------------+ |
| | 192.168.2.1 | | virbr0(bridge) | | |
| | aa:16:ea:a7:03:93 | | 192.168.122.1 | | |
| | | | 52:54:00:47:72:ce | | |
| | | | | | |
| +---------------------------------+ +--------------------------------+ | |
| | |
| | |
| Host +---------------------------------+
| | |
| | enP9p3s0 |
| | 10.10.0.9 |
| | |
| | 00:1b:21:e3:ec:50 |
| | |
+---------------------------------------------------------------------------------+---------------------------------+
Now you can browse the Internet.