sean@Supermicro-813MFTQC:~$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.5 LTS Release: 16.04 Codename: xenial sean@Supermicro-813MFTQC:~$ uname -r 4.4.0-131-generic
Use the Advanced Packaging Tool (APT) to update the list of available packages and their versions (update) and install newer versions of the packages installed (upgrade).
sean@Supermicro-813MFTQC:~$ sudo apt update && sudo apt upgrade -y
An Intel or AMD 64-bit CPU that has virtualization extensions, VT-x for Intel and AMD-V for AMD. Determine if your CPU supports the virtualiztion extensions and check for the svm, kvm, and lm flags. The vmx flag means that the CPU has VT-x (svm is for AMD) and lm is for 64-bit support.
sean@Supermicro-813MFTQC:~$ grep --color -Ew 'svm|vmx|lm' /proc/cpuinfo flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts ...
Verify KVM modules are loaded.
sean@Supermicro-813MFTQC:~$ lsmod | grep kvm kvm_intel 172032 0 kvm 548864 1 kvm_intel irqbypass 16384 1 kvm
In addition to virtualization extensions, you may need to enable Intel VT-d or AMD IOMMU (AMD-Vi) in the BIOS. These are required for direct PCI device assignment to VMs, for example, to assign a physical NIC from the hypervisor to the VM.
If running server-class VMs, then one core per vCPU is recommended. When counting cores, do not count the hyperthreaded cores on the Intel CPUs, just actual cores. However, if running desktop-class VMs or less CPU-intensive VMs, then it is safe to overcommit the CPU since the performance takes a back seat and priority changes to VM density per hypervisor more than performance.
Run the lscpu command to see the CPU topology.
sean@Supermicro-813MFTQC:~$ lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 40 On-line CPU(s) list: 0-39 Thread(s) per core: 2 Core(s) per socket: 10 Socket(s): 2 NUMA node(s): 2 Vendor ID: GenuineIntel CPU family: 6 Model: 79 Model name: Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz Stepping: 1 CPU MHz: 1200.203 CPU max MHz: 3100.0000 CPU min MHz: 1200.0000 BogoMIPS: 4401.33 Virtualization: VT-x L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 25600K NUMA node0 CPU(s): 0-9,20-29 NUMA node1 CPU(s): 10-19,30-39 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch epb invpcid_single intel_pt ibrs ibpb stibp kaiser tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdseed adx smap xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts
Simple rule of thumb to decide how much memory is need for a physical node is to add up all the memory that is planned to be assigned to VMs and add an additional 2 GB of RAM for the hypervisor to use.
Just like with physical CPUs, memory can be overcommitted for desktop-class VMs or test testing of VMs. View the available memory with the free command and the -g flag for GB.
sean@Supermicro-813MFTQC:~$ free -g
total used free shared buff/cache available
Mem: 251 0 250 0 0 250
Swap: 0 0 0
When considering storage space for the hypervisor, factor in the space required for the OS installation, Swap, and VMs disk usage.
If not planning to do any memory overcommit then 16 GB of swap space can be used for systems between 64 and 256 GB of RAM. If planning to do overcommit then additional swap space should be added. For example if the memory overcommit ration is 50% (.5) more than the available physical RAM then use the following formula: (RAM * 0.5) + Swap for OS = Swap space required for overcommitting. For example, with a system that has 256 GB RAM and planning to use a .5 overcommit ration, then Swap space required is (256 * .5) + 8 = 136 GB.
Swap space in Linux is used when the amount of physical memory (RAM) is full. If the system needs more memory resources and the RAM is full, inactive pages in memory are moved to the swap space.
Verify the configured Swap space and how much is available.
sean@Supermicro-813MFTQC:~$ swapon --show
NAME TYPE SIZE USED PRIO
/dev/sda5 partition 976M 0B -1
sean@Supermicro-813MFTQC:~$ free -m
total used free shared buff/cache available
Mem: 257848 599 255591 9 1658 256335
Swap: 975 0 975
The libvirtd daemon can be stoppped with sudo systemctl stop libvirtd and started with sudo systemctl start libvirtd commands. The libvirt service exposes an API to interact with qemu-kvm binary. Clients such as virsh and virt-manager use this API to communicate with qemu-kvm for VM life cycle management.
sean@Supermicro-813MFTQC:~$ libvirtd --version libvirtd (libvirt) 1.3.1
sean@Supermicro-813MFTQC:~$ sudo systemctl status libvirtd
● libvirt-bin.service - Virtualization daemon
Loaded: loaded (/lib/systemd/system/libvirt-bin.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2018-12-11 12:44:59 MST; 2h 30min ago
Docs: man:libvirtd(8)
http://libvirt.org
Main PID: 1536 (libvirtd)
Tasks: 18
Memory: 50.5M
CPU: 10.876s
CGroup: /system.slice/libvirt-bin.service
├─1536 /usr/sbin/libvirtd
├─1782 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─1783 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: started, version 2.75 cachesize 150
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inot
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq-dhcp[1782]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq-dhcp[1782]: DHCP, sockets bound exclusively to interface virbr0
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: reading /etc/resolv.conf
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: using nameserver 192.168.188.1#53
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: read /etc/hosts - 5 addresses
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq[1782]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Dec 11 12:45:10 Supermicro-813MFTQC dnsmasq-dhcp[1782]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Dec 11 15:15:43 Supermicro-813MFTQC systemd[1]: Started Virtualization daemon.
Perform sanity checks on KVM capabilities to validate that the host is conifugred in a suitable way to run the libvirt hypervisor drivers using KVM virtualization.
It is the hardware virtualization support that helps KVM (qemu-kvm) VMs to have direct access to the physical CPU and helps it reach nearly native performance.
sean@Supermicro-813MFTQC:~$ sudo virt-host-validate QEMU: Checking for hardware virtualization : PASS QEMU: Checking if device /dev/kvm exists : PASS QEMU: Checking if device /dev/kvm is accessible : PASS QEMU: Checking if device /dev/vhost-net exists : PASS QEMU: Checking if device /dev/net/tun exists : PASS QEMU: Checking for cgroup 'memory' controller support : PASS QEMU: Checking for cgroup 'memory' controller mount-point : PASS QEMU: Checking for cgroup 'cpu' controller support : PASS QEMU: Checking for cgroup 'cpu' controller mount-point : PASS QEMU: Checking for cgroup 'cpuacct' controller support : PASS QEMU: Checking for cgroup 'cpuacct' controller mount-point : PASS QEMU: Checking for cgroup 'devices' controller support : PASS QEMU: Checking for cgroup 'devices' controller mount-point : PASS QEMU: Checking for cgroup 'net_cls' controller support : PASS QEMU: Checking for cgroup 'net_cls' controller mount-point : PASS QEMU: Checking for cgroup 'blkio' controller support : PASS QEMU: Checking for cgroup 'blkio' controller mount-point : PASS QEMU: Checking for device assignment IOMMU support : PASS QEMU: Checking if IOMMU is enabled by kernel : WARN (IOMMU appears to be disabled in kernel. Add intel_iommu=on to kernel cmdline arguments) LXC: Checking for Linux >= 2.6.26 : PASS LXC: Checking for namespace ipc : PASS LXC: Checking for namespace mnt : PASS LXC: Checking for namespace pid : PASS LXC: Checking for namespace uts : PASS LXC: Checking for namespace net : PASS LXC: Checking for namespace user : PASS LXC: Checking for cgroup 'memory' controller support : PASS LXC: Checking for cgroup 'memory' controller mount-point : PASS LXC: Checking for cgroup 'cpu' controller support : PASS LXC: Checking for cgroup 'cpu' controller mount-point : PASS LXC: Checking for cgroup 'cpuacct' controller support : PASS LXC: Checking for cgroup 'cpuacct' controller mount-point : PASS LXC: Checking for cgroup 'devices' controller support : PASS LXC: Checking for cgroup 'devices' controller mount-point : PASS LXC: Checking for cgroup 'net_cls' controller support : PASS LXC: Checking for cgroup 'net_cls' controller mount-point : PASS LXC: Checking for cgroup 'freezer' controller support : PASS LXC: Checking for cgroup 'freezer' controller mount-point : PASS
virsh (virtualization shell) is a CLI for managing the VM and the hypervisor on a Linux system. It uses the libvirt management API and operates as an alternative to the GUI virt-manager.
virsh command classifications:
- Guest management (e.g.
start,stop) - Guest monitoring (e.g.
memstat,cpustat) - Host & hypervisor (e.g.
capabilities,nodeinfo) - Virtual networking (e.g.
net-list,net-define) - Storage management (e.g.
pool-list,pool-define) - Snapshot (e.g.
create-snapshot-as)
See the system hardware architecture, CPU topology, memory size, and so on.
sean@Supermicro-813MFTQC:~$ sudo virsh nodeinfo CPU model: x86_64 CPU(s): 40 CPU frequency: 1200 MHz CPU socket(s): 1 Core(s) per socket: 10 Thread(s) per core: 2 NUMA cell(s): 2 Memory size: 264037316 KiB
The virsh domcapabilities command displays an XML document describing the capabilities of qemu-kvm with respect to host and libvirt versions.
sean@Supermicro-813MFTQC:~$ virsh domcapabilities
<domainCapabilities>
<path>/usr/bin/kvm-spice</path>
<domain>qemu</domain>
<machine>pc-i440fx-xenial</machine>
<arch>x86_64</arch>
<vcpu max='255'/>
<os supported='yes'>
<loader supported='yes'>
<enum name='type'>
<value>rom</value>
<value>pflash</value>
</enum>
<enum name='readonly'>
<value>yes</value>
<value>no</value>
</enum>
</loader>
</os>
<devices>
<disk supported='yes'>
<enum name='diskDevice'>
<value>disk</value>
<value>cdrom</value>
<value>floppy</value>
<value>lun</value>
</enum>
<enum name='bus'>
<value>ide</value>
<value>fdc</value>
<value>scsi</value>
<value>virtio</value>
<value>usb</value>
</enum>
</disk>
<hostdev supported='yes'>
<enum name='mode'>
<value>subsystem</value>
</enum>
<enum name='startupPolicy'>
<value>default</value>
<value>mandatory</value>
<value>requisite</value>
<value>optional</value>
</enum>
<enum name='subsysType'>
<value>usb</value>
<value>pci</value>
<value>scsi</value>
</enum>
<enum name='capsType'/>
<enum name='pciBackend'/>
</hostdev>
</devices>
<features>
<gic supported='no'/>
</features>
</domainCapabilities>
Generate your key using the ssh-keygen command. Next you are asked for the directory in which to save your key files, defaulting to /home/${USER}/.ssh. Next you are asked for a passphrase, which is optional. If you need to change your password the command ssh-keygen -p can be used.
sean@X1-Carbon-6th-Gen:~$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/sean/.ssh/id_rsa): /home/sean/.ssh/mykey Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/sean/.ssh/mykey. Your public key has been saved in /home/sean/.ssh/mykey.pub. The key fingerprint is: SHA256:VYDv6V/iDWcGPIwSxM075PcIVSee8erekHjScvHuh80 sean@X1-Carbon-6th-Gen The key's randomart image is: +---[RSA 2048]----+ | ..+....+ .| | .o +... * | | .+.o o .| | o*+. . | | S..==oo | | .o .*.+ | | . * %+.| | .. #.+E| | .o o.=| +----[SHA256]-----+
Two files are created mykey and mykey.pub. The mykey file is the private key and should never leave the machine, be given to another user, or be stored on any external media. If the private key leaks out, then your keys can no longer be trusted. By default, the private key is owned by the user that created it, with rw permissions given only to its owner. The public key mykey.pub has more lenient permissions, being readable by everyone and writable by the owner.
sean@X1-Carbon-6th-Gen:~$ ls -l .ssh/ total 12K -rw-r--r-- 1 sean 444 Dec 11 12:50 known_hosts -rw------- 1 sean 1.8K Dec 11 17:24 mykey -rw-r--r-- 1 sean 404 Dec 11 17:24 mykey.pub
The public key is what gets copied to other servers to facilitate logging in via the SSH key-pair. When you log in to a server that has your public key, it checks that it’s a mathematical match to the private key on you machine, and if so then you are logged in. You’ll also be asked for your passphrase, if one was set. Use the ssh-copy-id command to transmit the public key to a target server.
sean@X1-Carbon-6th-Gen:~$ ssh-copy-id -i ~/.ssh/mykey.pub 192.168.188.4 /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/sean/.ssh/mykey.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys sean@192.168.188.4's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh '192.168.188.4'" and check to make sure that only the key(s) you wanted were added.
The contents of ~/.ssh/mykey.pub on your machine is copied into the ~/.ssh/authorized_keys file on the target server and SSH checks the contents of it when connecting, looking for a key that matches the private key (~/.ssh/mykey) on your local machine. If the two keys are a mathematical match, you are allowed access. If you set up a passphrase, you'll be asked to enter it in order to open your public key. With an SSH agent, you can actually cache your passphrase the first time you use it, so you won't be asked for it with every connection. To start the SSH agent, type eval $(ssh-agent).
sean@X1-Carbon-6th-Gen:~$ eval $(ssh-agent) Agent pid 20690
This command will start an SSH agent, that will continue to run in the background of your shell. Then, you can unlock your key for your agent to use with the ssh-add command.
sean@X1-Carbon-6th-Gen:~$ ssh-add ~/.ssh/mykey Enter passphrase for /home/sean/.ssh/mykey: Identity added: /home/sean/.ssh/mykey (/home/sean/.ssh/mykey)
sean@X1-Carbon-6th-Gen:~$ ssh 192.168.188.4 Welcome to Ubuntu 16.04.5 LTS (GNU/Linux 4.4.0-131-generic x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage 0 packages can be updated. 0 updates are security updates. New release '18.04.1 LTS' available. Run 'do-release-upgrade' to upgrade to it. *** System restart required *** Last login: Tue Dec 11 16:44:51 2018 from 192.168.188.223 sean@Supermicro-813MFTQC:~$
The virt-manager utility provides the ability to manage both local and remote KVM servers. To create a new connection to a remote external server, click on File and select Add Connection. A new screen will appear, where the connection details can be filled out. Check the Connect to remote host if not connecting to the local host (in which case you can leave the defaults). For the remote connection select the Method as SSH and fill out the Username, Hostname of the server where KVM was installed on and if Autoconnect if desired. For a remote connection to work the username included here will need to be able to access the server via SSH, have permissions to the hypervisor and the libvirtd unit must be running on the server. Press Connect to initiate the SSH connection and you may be prompted for the username password to your KVM server; type that in and press OK.
The Overview tab gives basic information on the libvirt connection URI, CPU, and memory usage pattern of the host system.
The Virtual Networks tab allows configuration of various types of virtual networks (VN) and monitor their status. There are three types of VNs:
- NAT'd
- Routed
- Isolated
NAT'd mode provides outbound network connectivity to attached VMs. This means that VMs can communicate with the outside network based on the network connectivity with the host. But none of the outside entities are able to communicate with the VMs.
By default a VM will get an IP address in the 192.168.122.0/24 network (virbr0 interface), within the range of .2 to .254.
Use the virsh net-list --all to list both active and inactive VNs. Without --all only active VNs are listed.
sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- default active yes yes
sean@Supermicro-813MFTQC:~$ virsh net-info default Name: default UUID: 74c996f0-5929-49f4-8d02-aabecb729901 Active: yes Persistent: yes Autostart: yes Bridge: virbr0
Virtual network configuration files are stored in /etc/libvirt/qemu/networks as XML files. The virsh net-destroy <VN> command will stop a VN while virsh net-start <VN> will start the VN.
:::danger
DO NOT use the the net-destroy when VMs are active using the VN as it will break network connectivity for the VM.
:::
sean@Supermicro-813MFTQC:~$ virsh net-dumpxml default
<network>
<name>default</name>
<uuid>74c996f0-5929-49f4-8d02-aabecb729901</uuid>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='virbr0' stp='on' delay='0'/>
<mac address='52:54:00:0c:0d:6f'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254'/>
</dhcp>
</ip>
</network>
sean@Supermicro-813MFTQC:~$ ifconfig virbr0
virbr0 Link encap:Ethernet HWaddr 52:54:00:0c:0d:6f
inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
With routed mode, the virtual switch is connected to the physical host LAN, passing guest network traffic back and forth without using NAT. The virtual switch sees the IP addresses in each packet, using that information when deciding what to do.
In this mode all VMs are in a subnet routed through the virtual switch. This on its own is not sufficient. because no other hosts on the physical network know this subnet exists or how to reach it. It is thus necessary to configure routers in the physical network (e.g. using a static route).
In this mode, guests connected to the virtual switch can communicate with each other, and with the host. However, their traffic will not pass outside of the host, nor can they receive traffic from outside the host.
The Storage Tab provides details of the storage pools available and is just a store for save VM disk images. Different sources such as directory and LVM can be used.
The default storage pool is in /var/lib/libvirt/images
Stop the libvirtd service as additional configuraiton will be done.
sean@Supermicro-813MFTQC:~$ sudo systemctl stop libvirtd
Ensure a group named kvm exist that will be used to allow members of the group to manage VMs. If the group is not created then perform the sudo groupadd kvm command.
sean@Supermicro-813MFTQC:~$ more /etc/group | grep kvm kvm:x:111:
Make the root user and the kvm group the owner of the /var/lib/libvirt/images directory.
sean@Supermicro-813MFTQC:~$ sudo chown root:kvm /var/lib/libvirt/images
Set the permissions of /var/lib/libvirt/images so that anyone in the kvm group will be able to modify its contents. Remember that d stands for directory and the permissions for user, group, and other are grouped in three sets with r (read), w write, x execute permissions.
sean@Supermicro-813MFTQC:~$ sudo chmod g+rw /var/lib/libvirt/images
sean@Supermicro-813MFTQC:~$ sudo ls -la /var/lib/libvirt | grep images drwxrwx--x 2 root kvm 4096 May 23 2018 images
Set the primary user account in use on the server to be a member of the kvm group in order to manage VMs without switching to root first. Log out and log in again after executing the command, so the changes take effect.
sean@Supermicro-813MFTQC:~$ sudo usermod -aG kvm sean
sean@Supermicro-813MFTQC:~$ more /etc/group | grep kvm kvm:x:111:sean
Start the libvirtd service and check the status for errors.
sean@Supermicro-813MFTQC:~$ sudo systemctl start libvirtd
sean@Supermicro-813MFTQC:~$ sudo systemctl status libvirtd
● libvirt-bin.service - Virtualization daemon
Loaded: loaded (/lib/systemd/system/libvirt-bin.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2018-12-11 18:36:27 MST; 14s ago
Docs: man:libvirtd(8)
http://libvirt.org
Main PID: 1924 (libvirtd)
Tasks: 19
Memory: 42.0M
CPU: 14.342s
CGroup: /system.slice/libvirt-bin.service
├─1782 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
├─1783 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
└─1924 /usr/sbin/libvirtd
Dec 11 18:36:27 Supermicro-813MFTQC systemd[1]: Starting Virtualization daemon...
Dec 11 18:36:27 Supermicro-813MFTQC systemd[1]: Started Virtualization daemon.
Dec 11 18:36:41 Supermicro-813MFTQC dnsmasq[1782]: read /etc/hosts - 5 addresses
Dec 11 18:36:41 Supermicro-813MFTQC dnsmasq[1782]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
Dec 11 18:36:41 Supermicro-813MFTQC dnsmasq-dhcp[1782]: read /var/lib/libvirt/dnsmasq/default.hostsfile
Copy the CentOS minimal ISO file to the /var/lib/libvirt/images/iso, which installs only a very small base set of packages and is the smallest download of CentOS available.
sean@X1-Carbon-6th-Gen:~/Downloads/Software$ scp CentOS-7-x86_64-Minimal-1804.iso 192.168.188.4:/var/lib/libvirt/images/. CentOS-7-x86_64-Minimal-1804.iso 100% 906MB 8.1MB/s 01:51
In virt-manager, right-click your server connection and click on New to start the creation of a VM. The default selection will be on Local install media (ISO image or CDROM); leave this selection and click on Forward.
Click on Browse to open up another window where you can select an ISO image under the default filesystem directory. Click on Choose Volume and then Forward.
Next, allocate RAM and CPU resources to the VM. For most Linux distributions with no graphical user interface, 512 MB is plenty (unless your workload demands more). Click on Forward.
Next, allocate free disk space for the VM's virtual hard disk. This space won't be used up all at once; by default, KVM utilizes thin provisioning that basically just fills up the virtual disk as the VM needs space. Click on Forward.
Lastly, name the VM. This won't be the hostname of the virtual machine; it's just the name you'll see when you see the VM listed in virt-manager. Selecting Customize configuration before install box will open another wizard that provides the ability to add, remove, and configure VM hardware settings. Click on Finish, the VM will start and it will automatically boot into the install ISO that was attached to the VM near the beginning of the process.
When you click on the VM window, it may steal the keyboard and mouse and dedicate it to the window. Press Ctrl + Alt at the same time to release this control and regain full control of the keyboard and mouse.
The default MAC address range used by libvirt is
52:54:00.
sean@Supermicro-813MFTQC:~$ virsh net-dhcp-leases default Expiry Time MAC address Protocol IP address Hostname Client ID or DUID ------------------------------------------------------------------------------------------------------------------- 2018-12-11 20:13:26 52:54:00:72:38:e0 ipv4 192.168.122.17/24 centos7 -
sean@Supermicro-813MFTQC:~$ ssh 192.168.122.17 The authenticity of host '192.168.122.17 (192.168.122.17)' can't be established. ECDSA key fingerprint is SHA256:ojUEdhup2GDnJ+hq4UiFzOEOkMZRgb+8M+lmeeXA8CU. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '192.168.122.17' (ECDSA) to the list of known hosts. sean@192.168.122.17's password: Last login: Tue Dec 11 19:14:14 2018 [sean@centos7 ~]$
[sean@centos7 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:72:38:e0 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.17/24 brd 192.168.122.255 scope global noprefixroute dynamic eth0
valid_lft 3171sec preferred_lft 3171sec
inet6 fe80::f484:1b6:1edb:a0f2/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[sean@centos7 ~]$ ping google.com -c 3 PING google.com (172.217.11.238) 56(84) bytes of data. 64 bytes from den02s01-in-f14.1e100.net (172.217.11.238): icmp_seq=1 ttl=55 time=36.0 ms 64 bytes from den02s01-in-f14.1e100.net (172.217.11.238): icmp_seq=2 ttl=55 time=25.3 ms 64 bytes from den02s01-in-f14.1e100.net (172.217.11.238): icmp_seq=3 ttl=55 time=25.3 ms --- google.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 25.307/28.915/36.085/5.071 ms
virt-install is an alternative CLI tool that is scripting-friendly and supports graphical installations that can be used to setup guest and then start the install process.
Before starting the OS install using the virt-install command, it is necessary to create a virtual disk using the qemu-img command.
:::info Notice this is performed on a Ubuntu desktop. :::
sean@X1-Carbon-6th-Gen:~$ sudo qemu-img create -f raw -o size=9G /var/lib/libvirt/qemu/centos7.img Formatting '/var/lib/libvirt/qemu/centos7.img', fmt=raw size=9663676416
Start the installation with the virt-install command and it will launch virt-viewer.
sean@X1-Carbon-6th-Gen:~$ sudo virt-install \ > --name centos7.0 \ > --ram 512 \ > --disk path=/var/lib/libvirt/qemu/centos7.img \ > --vcpus 1 \ > --os-type Linux \ > --os-variant CentOS7.0 \ > --network bridge=virbr0 \ > --graphics vnc,port=5999 \ > --console pty,target_type=serial \ > --cdrom /var/lib/libvirt/images/CentOS-7-x86_64-Minimal-1804.iso
The main component of libvirt networking is the virtual switch, also known as the ++bridge++. Interfaces connecting VMs to a bridge are called ++TAP devices++. The TAP interface is part of the TUN/TAP implementation available in the Linux kernal. TAP (network tap) simulates a link layer device and operates at OSI L2 for Ethernet frames. TUN stands for "tunnel" and operates at OSI L3 for IP packets.
The brctl command is provided by the package bridge-utils. Create a bridge called "test". The brctl show command will list all available bridges on the server along with the ID of the bridge, STP status, and interfaces attached to it.
sean@Supermicro-813MFTQC:~$ sudo brctl addbr test sean@Supermicro-813MFTQC:~$ brctl show bridge name bridge id STP enabled interfaces test 8000.000000000000 no virbr0 8000.5254000c0d6f yes virbr0-nic
A Linux bridge will show up as a network device using the ip command.
sean@Supermicro-813MFTQC:~$ ip -c link show test
9: test: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 56:7c:eb:34:d7:19 brd ff:ff:ff:ff:ff:ff
Likewise, ifconfig can be used to check and configure network settings for a Linux bridge. ifconfig is relatively easy to read and understand but is not as feature-rich as the ip command.
sean@Supermicro-813MFTQC:~$ ifconfig test
test Link encap:Ethernet HWaddr 56:7c:eb:34:d7:19
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Create a tap device named "vm-nic" using the TUN/TAP device module.
sean@Supermicro-813MFTQC:~$ ip tuntap add dev vm-vnic mode tap
sean@Supermicro-813MFTQC:~$ ip -c link show vm-vnic
10: vm-vnic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether d6:4e:9a:24:99:ae brd ff:ff:ff:ff:ff:ff
Add the tap device "vm-vnic" to the bridge "test".
sean@Supermicro-813MFTQC:~$ sudo brctl addif test vm-vnic sean@Supermicro-813MFTQC:~$ brctl show bridge name bridge id STP enabled interfaces test 8000.d64e9a2499ae no vm-vnic virbr0 8000.5254000c0d6f yes virbr0-nic
Remove the "vm-nic" tap device from the "test" bridge.
sean@Supermicro-813MFTQC:~$ sudo brctl delif test vm-vnic sean@Supermicro-813MFTQC:~$ brctl show test bridge name bridge id STP enabled interfaces test 8000.000000000000 no
Once the "vm-vnic" is removed from the bridge, remote the tap device using the ip command.
sean@Supermicro-813MFTQC:~$ ip tuntap del dev vm-vnic mode tap
Finally, remove the "test" bridge.
sean@Supermicro-813MFTQC:~$ sudo brctl delbr test; echo $? 0
Use the virsh command to build an isolated network by creating an XML file with the following contents and save it as "isolated.xml". Here:
<network>: is used for defining the virtual net.<name>: is used for defining the name of the virtual net an is called "isolated".
<network>
<name>isolated</name>
</network>
To create the network using the XML file use the virsh net-define command followed by the path of the XML file. Here libvirt added additional parameters including:
<uuid>: unique ID for a bridge.<bridge>: is used to define bridge details such as the name of the bridge isvirbr1, with STP on, and a delay of 0. These are the same options that can be controlled with thebrctlcommand options with STP set bystpand delay bysetfd.<mac>: is the MAC address assigned to the bridge.
sean@Supermicro-813MFTQC:~$ virsh net-define isolated.xml Network isolated defined from isolated.xml sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- default active yes yes isolated inactive no yes
View the XML file that libvirt created using the virsh net-dumpxml command.
sean@Supermicro-813MFTQC:~$ virsh net-dumpxml isolated <network> <name>isolated</name> <uuid>2213fd59-2603-496a-9cf1-7f27d7fc220f</uuid> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:67:27:65'/> </network>
The configuration file is stored in /etc/libvirt/qemu/networks/ as an XML file with the given name of the virtual network.
sean@Supermicro-813MFTQC:~$ sudo cat /etc/libvirt/qemu/networks/isolated.xml <!-- WARNING: THIS IS AN AUTO-GENERATED FILE. CHANGES TO IT ARE LIKELY TO BE OVERWRITTEN AND LOST. Changes to this xml configuration should be made using: virsh net-edit isolated or other application using the libvirt API. --> <network> <name>isolated</name> <uuid>2213fd59-2603-496a-9cf1-7f27d7fc220f</uuid> <bridge name='virbr1' stp='on' delay='0'/> <mac address='52:54:00:67:27:65'/> </network>
Activate the isolated network using the virsh net-start command and auto-start with the virsh net-autostart command.
sean@Supermicro-813MFTQC:~$ virsh net-start isolated Network isolated started sean@Supermicro-813MFTQC:~$ virsh net-autostart isolated Network isolated marked as autostarted sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- default active yes yes isolated active yes yes
A virtio vNIC can be added while a VM is running and will be ready to use inside the VM immediately.
Get the details of the current vNIC attached to VM "c1" using the virsh domiflist command.
sean@Supermicro-813MFTQC:~$ virsh domiflist c2 Interface Type Source Model MAC ------------------------------------------------------- vnet1 network default virtio 52:54:00:46:1f:75
Attach a new vNIC to VM "c2" using the virsh attach-interface command. Two new options include:
--config: makes the change persistent in the next startup of the VM.--live: informs libvirt a NIC is being attached to a live VM.
sean@Supermicro-813MFTQC:~$ virsh attach-interface --domain c2 --source isolated --type network --model virtio --config --live Interface attached successfully sean@Supermicro-813MFTQC:~$ virsh domiflist c2 Interface Type Source Model MAC ------------------------------------------------------- vnet1 network default virtio 52:54:00:46:1f:75 vnet2 network isolated virtio 52:54:00:11:d7:0b
The virbr1-nic interface is created by libvirt when it starts virbr1. The purpose of this interface is to provide a consistent and reliable MAC address for the bridge.
sean@Supermicro-813MFTQC:~$ brctl show virbr1 bridge name bridge id STP enabled interfaces virbr1 8000.525400672765 yes virbr1-nic vnet2
Connect VM "c1" to VM "c2" over the isolated network and assign IPs to the newly added interfaces and verify that reachability is established with the ping command.
sean@Supermicro-813MFTQC:~$ virsh net-dhcp-leases default Expiry Time MAC address Protocol IP address Hostname Client ID or DUID ------------------------------------------------------------------------------------------------------------------- 2018-12-11 23:11:45 52:54:00:46:1f:75 ipv4 192.168.122.14/24 c2 - 2018-12-11 23:16:07 52:54:00:8a:42:bb ipv4 192.168.122.162/24 c1 -
[sean@c1 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth1 TYPE="Ethernet" BOOTPROTO="static" IPV4_FAILURE_FATAL="no" NAME="eth1" DEVICE="eth1" ONBOOT="yes" IPADDR="192.168.1.100" PREFIX="24"
[sean@c1 ~]$ systemctl restart network ==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units === Authentication is required to manage system services or units. Authenticating as: Sean (sean) Password: ==== AUTHENTICATION COMPLETE ===
[sean@c1 ~]$ systemctl status network
● network.service - LSB: Bring up/down networking
Loaded: loaded (/etc/rc.d/init.d/network; bad; vendor preset: disabled)
Active: active (exited) since Tue 2018-12-11 22:47:20 MST; 3s ago
Docs: man:systemd-sysv-generator(8)
Process: 9707 ExecStop=/etc/rc.d/init.d/network stop (code=exited, status=0/SUCCESS)
Process: 9944 ExecStart=/etc/rc.d/init.d/network start (code=exited, status=0/SUCCESS)
Dec 11 22:47:20 c1 systemd[1]: Starting LSB: Bring up/down networking...
Dec 11 22:47:20 c1 network[9944]: Bringing up loopback interface: [ OK ]
Dec 11 22:47:20 c1 network[9944]: Bringing up interface eth0: Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/21)
Dec 11 22:47:20 c1 network[9944]: [ OK ]
Dec 11 22:47:20 c1 network[9944]: Bringing up interface eth1: Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/22)
Dec 11 22:47:20 c1 network[9944]: [ OK ]
Dec 11 22:47:20 c1 systemd[1]: Started LSB: Bring up/down networking.
[sean@c1 ~]$ ip -c addr show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:18:6b:4b brd ff:ff:ff:ff:ff:ff
inet 192.168.1.100/24 brd 192.168.1.255 scope global noprefixroute eth1
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe18:6b4b/64 scope link
valid_lft forever preferred_lft forever
[sean@c1 ~]$ ping 192.168.1.200 -c 3 PING 192.168.1.200 (192.168.1.200) 56(84) bytes of data. 64 bytes from 192.168.1.200: icmp_seq=1 ttl=64 time=0.855 ms 64 bytes from 192.168.1.200: icmp_seq=2 ttl=64 time=0.400 ms 64 bytes from 192.168.1.200: icmp_seq=3 ttl=64 time=0.412 ms --- 192.168.1.200 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.400/0.555/0.855/0.213 ms
[sean@c1 ~]$ ip n show dev eth1 192.168.1.200 lladdr 52:54:00:11:d7:0b DELAY
Log into the "c2" VM and verify the MAC address.
[sean@c2 ~]$ ip -c addr show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:11:d7:0b brd ff:ff:ff:ff:ff:ff
inet 192.168.1.200/24 brd 192.168.1.255 scope global noprefixroute eth1
valid_lft forever preferred_lft forever
inet6 fe80::5054:ff:fe11:d70b/64 scope link
valid_lft forever preferred_lft forever
Create an XML configuration file called "routed.xml".
<network>
<name>routed</name>
<forward dev='eno1' mode='route'>
<interface dev='eno1'/>
</forward>
<ip address='192.168.2.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.2.128' end='192.168.2.254'/>
</dhcp>
</ip>
</network>
sean@Supermicro-813MFTQC:~$ virsh net-define routed.xml Network routed defined from routed.xml sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- default active yes yes isolated active yes yes routed inactive no yes
sean@Supermicro-813MFTQC:~$ virsh net-start routed Network routed started sean@Supermicro-813MFTQC:~$ virsh net-autostart routed Network routed marked as autostarted sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- default active yes yes isolated active yes yes routed active yes yes
sean@Supermicro-813MFTQC:~$ virsh net-dumpxml routed
<network connections='1'>
<name>routed</name>
<uuid>eebe92aa-6e01-4559-8964-57b02d11395d</uuid>
<forward dev='eno1' mode='route'>
<interface dev='eno1'/>
</forward>
<bridge name='virbr2' stp='on' delay='0'/>
<mac address='52:54:00:e3:bd:c4'/>
<ip address='192.168.2.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.2.128' end='192.168.2.254'/>
</dhcp>
</ip>
</network>
Check that network is marked as persistent and autostart.
sean@Supermicro-813MFTQC:~$ virsh net-info routed Name: routed UUID: eebe92aa-6e01-4559-8964-57b02d11395d Active: yes Persistent: yes Autostart: yes Bridge: virbr2
On the "c1" VM on the host server (Supermicro-813MFTQC) setup the eth1 interface to belong to the routed network (192.168.2/24) and set it for DHCP. Then turn down the eth0 interface that is attached to the default NAT'd network.
[sean@c1 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth1 TYPE="Ethernet" BOOTPROTO="dhcp" IPV4_FAILURE_FATAL="no" NAME="eth1" DEVICE="eth1" ONBOOT="yes"
[sean@c1 ~]$ systemctl restart network ==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units === Authentication is required to manage system services or units. Authenticating as: Sean (sean) Password: ==== AUTHENTICATION COMPLETE ===
[sean@c1 ~]$ sudo ifdown eth0 Device 'eth0' successfully disconnected.
[sean@c1 ~]$ ip -c address show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:18:6b:4b brd ff:ff:ff:ff:ff:ff
inet 192.168.2.133/24 brd 192.168.2.255 scope global noprefixroute dynamic eth1
valid_lft 3493sec preferred_lft 3493sec
inet6 fe80::5054:ff:fe18:6b4b/64 scope link
valid_lft forever preferred_lft forever
[sean@c1 ~]$ ip route default via 192.168.2.1 dev eth1 proto dhcp metric 101 192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.133 metric 101
sean@Supermicro-813MFTQC:~$ virsh net-dhcp-leases routed Expiry Time MAC address Protocol IP address Hostname Client ID or DUID ------------------------------------------------------------------------------------------------------------------- 2018-12-12 21:12:16 52:54:00:18:6b:4b ipv4 192.168.2.133/24 c1 -
Setup a static route for the "routed" network 192.168.2/24 back to the host server (Supermicro-813MFTQC) on the LAN at 192.168.188.4 on the target LAN device (X1-Carbon-6th-Gen) that is in the 192.168.188/24 network.
sean@X1-Carbon-6th-Gen:~$ ip -c address show dev wlp2s0
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 74:e5:f9:65:08:79 brd ff:ff:ff:ff:ff:ff
inet 192.168.188.223/24 brd 192.168.188.255 scope global dynamic noprefixroute wlp2s0
valid_lft 41881sec preferred_lft 41881sec
inet6 fe80::2232:36a9:1595:7a59/64 scope link noprefixroute
valid_lft forever preferred_lft forever
sean@X1-Carbon-6th-Gen:~$ sudo ip route add 192.168.2.0/24 via 192.168.188.4
sean@X1-Carbon-6th-Gen:~$ ip route get 192.168.2.0/24
192.168.2.0 via 192.168.188.4 dev wlp2s0 src 192.168.188.223 uid 1000
cache
Perform a ping from VM "c1" on the host server (Supermicro-813MFTQC) to 192.168.188.223 (X1-Carbon-6th-Gen) in order to verify reachability. Perform a tcpdump on the target device (X1-Carbon-6th-Gen) to capture the ICMP pings.
[sean@c1 ~]$ ping 192.168.188.223 -c 3 PING 192.168.188.223 (192.168.188.223) 56(84) bytes of data. 64 bytes from 192.168.188.223: icmp_seq=1 ttl=63 time=1.06 ms 64 bytes from 192.168.188.223: icmp_seq=2 ttl=63 time=1.24 ms 64 bytes from 192.168.188.223: icmp_seq=3 ttl=63 time=1.37 ms --- 192.168.188.223 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 1.064/1.226/1.375/0.130 ms
sean@X1-Carbon-6th-Gen:~$ sudo tcpdump -nni wlp2s0 icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on wlp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes 19:40:09.776932 IP 192.168.2.133 > 192.168.188.223: ICMP echo request, id 3898, seq 1, length 64 19:40:09.777009 IP 192.168.188.223 > 192.168.2.133: ICMP echo reply, id 3898, seq 1, length 64 19:40:10.778410 IP 192.168.2.133 > 192.168.188.223: ICMP echo request, id 3898, seq 2, length 64 19:40:10.778492 IP 192.168.188.223 > 192.168.2.133: ICMP echo reply, id 3898, seq 2, length 64 19:40:11.779847 IP 192.168.2.133 > 192.168.188.223: ICMP echo request, id 3898, seq 3, length 64 19:40:11.779927 IP 192.168.188.223 > 192.168.2.133: ICMP echo reply, id 3898, seq 3, length 64 ^C 6 packets captured 6 packets received by filter 0 packets dropped by kernel
Likewise confirm reachability from the target (X1-Carbon-6th-Gen) device, which is on the 192.168.188/24 network back to VM "c1" (192.168.2.133) on the server host (Supermicro-813MFTQC).
sean@X1-Carbon-6th-Gen:~$ ping 192.168.2.133 -c 3 PING 192.168.2.133 (192.168.2.133) 56(84) bytes of data. 64 bytes from 192.168.2.133: icmp_seq=1 ttl=63 time=1.79 ms 64 bytes from 192.168.2.133: icmp_seq=2 ttl=63 time=3.65 ms 64 bytes from 192.168.2.133: icmp_seq=3 ttl=63 time=1.38 ms --- 192.168.2.133 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.389/2.279/3.650/0.983 ms
Edit the routed virtual network so that packets from the VMs can be forwarded to any interface available on the host based on IP routes rules specified on the host.
The VM "c1" was stopped at this point in order to manipulate the "routed" network. Then it is okay to stop the VN using the virsh net-destroy command.
sean@Supermicro-813MFTQC:~$ virsh net-destroy routed Network routed destroyed
Edit the network using virsh net-edit command. A temporary copy of the configuration file is placed in the /tmp directory. The <forward> tag will be modified. Verify the changes with the virsh net-dumpxml command.
sean@Supermicro-813MFTQC:~$ virsh net-edit routed Select an editor. To change later, run 'select-editor'. 1. /bin/ed 2. /bin/nano <---- easiest 3. /usr/bin/vim.basic 4. /usr/bin/vim.tiny Choose 1-4 [2]: 2 Network routed XML configuration edited.
sean@Supermicro-813MFTQC:~$ virsh net-dumpxml routed
<network>
<name>routed</name>
<uuid>eebe92aa-6e01-4559-8964-57b02d11395d</uuid>
<forward mode='route'/>
<bridge name='virbr2' stp='on' delay='0'/>
<mac address='52:54:00:e3:bd:c4'/>
<ip address='192.168.2.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.2.128' end='192.168.2.254'/>
</dhcp>
</ip>
</network>
Restart the VN using the virh net-start command.
sean@Supermicro-813MFTQC:~$ virsh net-start routed Network routed started
The "c1" VM was restarted and verification of reachability was performed again to the target device (X1-Carbon-6th-Gen).
[sean@c1 ~]$ sudo ifdown eth0 Device 'eth0' successfully disconnected.
[sean@c1 ~]$ ip route default via 192.168.2.1 dev eth1 proto dhcp metric 101 192.168.2.0/24 dev eth1 proto kernel scope link src 192.168.2.133 metric 101
[sean@c1 ~]$ ping 192.168.188.223 -c 3 PING 192.168.188.223 (192.168.188.223) 56(84) bytes of data. 64 bytes from 192.168.188.223: icmp_seq=1 ttl=63 time=1.35 ms 64 bytes from 192.168.188.223: icmp_seq=2 ttl=63 time=1.62 ms 64 bytes from 192.168.188.223: icmp_seq=3 ttl=63 time=1.24 ms --- 192.168.188.223 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 1.246/1.410/1.629/0.161 ms
Likewise from the target device (X1-Carbon-6th-Gen), verify reachability to the "c1" VM at 192.168.2.133 (routed network) on the host server (Supermicro-813MFTQC) that is on the LAN at 192.168.188.4.
Create the br0 bridge with the eno2 interface and setup so the br0 can be used while creating VM network interfaces. This way VMs will acquire a DHCP address from the LAN that eno2 is connected to.
sean@Supermicro-813MFTQC:~$ cat /etc/network/interfaces # This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). source /etc/network/interfaces.d/* # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eno1 iface eno1 inet dhcp # IP address = 192.168.188.4 # Setup bridge interface on eno2 auto eno2 iface eno2 inet manual auto br0 iface br0 inet dhcp hwaddress ether ac:1f:6b:61:5e:dd # IP address = 192.168.188.5 bridge_ports eno2 bridge_stp on bridge_fd 0
sean@Supermicro-813MFTQC:~$ sudo systemctl restart networking
sean@Supermicro-813MFTQC:~$ brctl show bridge name bridge id STP enabled interfaces br0 8000.ac1f6b615edd yes eno2 virbr0 8000.5254000c0d6f yes virbr0-nic
sean@Supermicro-813MFTQC:~$ ip -c address show dev br0
41: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ac:1f:6b:61:5e:dd brd ff:ff:ff:ff:ff:ff
inet 192.168.188.5/24 brd 192.168.188.255 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::ae1f:6bff:fe61:5edd/64 scope link
valid_lft forever preferred_lft forever
Create an XML configuration file called “br0.xml”.
<network>
<name>br0</name>
<forward mode="bridge"/>
<bridge name="br0"/>
</network>
sean@Supermicro-813MFTQC:~$ virsh net-define br0.xml Network br0 defined from br0.xml sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- br0 inactive no yes default active yes yes sean@Supermicro-813MFTQC:~$ virsh net-start br0 Network br0 started sean@Supermicro-813MFTQC:~$ virsh net-autostart br0 Network br0 marked as autostarted sean@Supermicro-813MFTQC:~$ virsh net-list --all Name State Autostart Persistent ---------------------------------------------------------- br0 active yes yes default active yes yes sean@Supermicro-813MFTQC:~$ virsh net-dumpxml br0 <network> <name>br0</name> <uuid>b074bb07-13a2-4847-93bd-690be8c27c7c</uuid> <forward mode='bridge'/> <bridge name='br0'/> </network>
sean@Supermicro-813MFTQC:~$ brctl show bridge name bridge id STP enabled interfaces br0 8000.ac1f6b615edd yes eno2 vnet0 virbr0 8000.5254000c0d6f yes virbr0-nic
[sean@c1 ~]$ ip -c address show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:8a:42:bb brd ff:ff:ff:ff:ff:ff
inet 192.168.188.201/24 brd 192.168.188.255 scope global noprefixroute dynamic eth0
valid_lft 42882sec preferred_lft 42882sec
inet6 fe80::ccf3:ee70:4db8:17f7/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[sean@c1 ~]$ ping google.com -c 3 PING google.com (74.125.138.139) 56(84) bytes of data. 64 bytes from yi-in-f139.1e100.net (74.125.138.139): icmp_seq=1 ttl=46 time=104 ms 64 bytes from yi-in-f139.1e100.net (74.125.138.139): icmp_seq=2 ttl=46 time=93.3 ms 64 bytes from yi-in-f139.1e100.net (74.125.138.139): icmp_seq=3 ttl=46 time=96.4 ms --- google.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 93.397/98.078/104.394/4.635 ms
MacVTap is used when users in the LAN want to access the VM, but you don't want to create a normal Linux bridge. This connection is not used in production systems and is mostly used on a workstation-type station.
sean@X1-Carbon-6th-Gen:~$ ping 192.168.188.201 -c 3 PING 192.168.188.201 (192.168.188.201) 56(84) bytes of data. 64 bytes from 192.168.188.201: icmp_seq=1 ttl=64 time=124 ms 64 bytes from 192.168.188.201: icmp_seq=2 ttl=64 time=7.63 ms 64 bytes from 192.168.188.201: icmp_seq=3 ttl=64 time=5.29 ms --- 192.168.188.201 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 5.293/45.693/124.156/55.489 ms
[sean@c1 ~]$ ip -c address show dev eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:8a:42:bb brd ff:ff:ff:ff:ff:ff
inet 192.168.188.201/24 brd 192.168.188.255 scope global noprefixroute dynamic eth0
valid_lft 42523sec preferred_lft 42523sec
inet6 fe80::ccf3:ee70:4db8:17f7/64 scope link noprefixroute
valid_lft forever preferred_lft forever