diff --git a/kubevirt-patch/README.md b/kubevirt-patch/README.md index 725339c..6725d47 100644 --- a/kubevirt-patch/README.md +++ b/kubevirt-patch/README.md @@ -352,6 +352,18 @@ The original idea to build within the Centos container comes from this [link](ht sudo ctr -n k8s.io images import virt-launcher.tar sudo ctr -n k8s.io images import virt-operator.tar ``` + Alternatively + ```sh + sudo k3s ctr i import virt-operator.tar + sudo k3s ctr i import virt-api.tar + sudo k3s ctr i import virt-controller.tar + sudo k3s ctr i import virt-handler.tar + sudo k3s ctr i import virt-launcher.tar + sudo k3s ctr i import sidecar-shim.tar + + sudo k3s ctr i import device-plugin.tar + sudo k3s ctr i import busybox.tar + ``` 7. Verify the images are imported correctly ```sh sudo crictl images | grep localhost @@ -404,3 +416,50 @@ The original idea to build within the Centos container comes from this [link](ht NAME AGE PHASE kubevirt.kubevirt.io/kubevirt 19d Deployed ``` +10. Enable Virt-Handler to discover Graphics VFs + Update KubeVirt custom resource configuration to enable virt-handler to discover graphics VFs on the host. All discovered VFs will be published as *allocatable* resource + + **Update Graphics Device ID in `kubevirt-cr-gfx-sriov.yaml` if not found** + - Read the Device ID of Intel Graphics Card from Host, Ex: for RPL + ```sh + $ cat /sys/devices/pci0000\:00/0000\:00\:02.0/device + + 0xa7a0 + ``` + - Add the Device ID in `pciHostDevices` section + ```yaml + - pciVendorSelector: "8086:a7a0" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + ``` + + Apply the YAML changes + ```sh + kubectl apply -f manifests/kubevirt-cr-gfx-sriov.yaml + ``` + + **Check for presence of `intel.com/sriov-gpudevices` resource** + + ```sh + kubectl describe nodes + ``` + Output: + ```sh + Capacity: + intel.com/sriov-gpudevice: 7 + Allocatable: + intel.com/sriov-gpudevice: 7 + Allocated resources: + Resource Requests Limits + -------- -------- ------ + intel.com/sriov-gpudevice 0 0 + ``` + > [!Note] + > Please wait for all virt-handler pods to complete restarts\ + > The value of **Requests** and **Limits** will increase upon successful resource allocation to running pods/VMs + +11. Install CDI + ```sh + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-operator.yaml + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-cr.yaml + ``` diff --git a/kubevirt-patch/desktop-virtualization-k3s.spec b/kubevirt-patch/desktop-virtualization-k3s.spec index e9c7059..7536fcf 100644 --- a/kubevirt-patch/desktop-virtualization-k3s.spec +++ b/kubevirt-patch/desktop-virtualization-k3s.spec @@ -1,32 +1,50 @@ -Name: desktop-virtualization-k3s -Version: v1.0 +Name: intel-desktop-virtualization-k3s +Version: 0.1 Release: 1%{?dist} -Summary: Installs Kubevirt (v1.5.0 enabled with GTK libarary support and Intel SR-IOV patched QEMU in Virt-Launcher) and Device Plugin(v1) for enabling support of local GTK display using pre-built container tar files - -License: APACHE 2.0 -Source0: kubevirt.tar.gz -Source1: dv-device-plugin.tar.gz +Vendor: Intel Corporation +Distribution: Edge Microvisor Toolkit +URL: https://github.com/open-edge-platform/edge-desktop-virtualization +Summary: Provides Kubevirt (enabled with GTK libarary support and Intel SR-IOV patched QEMU in Virt-Launcher) and IDV Device Plugin for enabling support of local GTK display using pre-built container tar files +License: Apache-2.0 +Source0: https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-kubevirt-v0.1.tar.gz +Source1: https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-device-plugin-v0.1.tar.gz BuildArch: x86_64 +Requires: k3s %description -This RPM Installs Kubevirt (v1.5.0 enabled with GTK libarary support and Intel SR-IOV patched QEMU in Virt-Launcher) and Device Plugin(v1) for enabling support of local GTK display using pre-built container tar files +Provides Kubevirt (enabled with GTK libarary support and Intel SR-IOV patched QEMU in Virt-Launcher) and IDV Device Plugin for enabling support of local GTK display using pre-built container tar files %prep +tar -xzf %{SOURCE0} -C . +tar -xzf %{SOURCE1} -C . %build %install -mkdir -p %{buildroot}/usr/share/%{name} -cp -a %{SOURCE0} %{buildroot}/usr/share/%{name}/ -cp -a %{SOURCE1} %{buildroot}/usr/share/%{name}/ +mkdir -p %{buildroot}%{_sharedstatedir}/rancher/k3s/agent/images/ +cp *.tar %{buildroot}%{_sharedstatedir}/rancher/k3s/agent/images/ + +mkdir -p %{buildroot}%{_sharedstatedir}/rancher/k3s/server/manifests/ +cp *.yaml %{buildroot}%{_sharedstatedir}/rancher/k3s/server/manifests/ %files -/usr/share/%{name}/kubevirt.tar.gz -/usr/share/%{name}/dv-device-plugin.tar.gz +%{_sharedstatedir}/rancher/k3s/agent/images/virt-api.tar +%{_sharedstatedir}/rancher/k3s/agent/images/virt-controller.tar +%{_sharedstatedir}/rancher/k3s/agent/images/virt-handler.tar +%{_sharedstatedir}/rancher/k3s/agent/images/virt-launcher.tar +%{_sharedstatedir}/rancher/k3s/agent/images/virt-operator.tar +%{_sharedstatedir}/rancher/k3s/agent/images/busybox.tar +%{_sharedstatedir}/rancher/k3s/agent/images/device-plugin.tar +%{_sharedstatedir}/rancher/k3s/agent/images/sidecar-shim.tar +%{_sharedstatedir}/rancher/k3s/server/manifests/device-plugin.yaml +%{_sharedstatedir}/rancher/k3s/server/manifests/kubevirt-cr.yaml +%{_sharedstatedir}/rancher/k3s/server/manifests/kubevirt-operator.yaml + %post %changelog -* Thu Jun 5 2025 D M, Karthik - v1.0 -- Initial version of Kubevirt v1.5.0 with Display Virtualization and GTK library support -- Initial version of Device Plugin v1 to support Display Virtualization on local display +* Thu Jun 5 2025 D M, Karthik - 0.1-1 +- Original version for Edge Microvisor Toolkit. License verified. +- Pre-release version of Kubevirt v1.5.0 with Display Virtualization and GTK library support identified as v1.5.0_DV +- Pre-release version of Device Plugin v1 to support Display Virtualization on local display diff --git a/kubevirt-patch/tiber_mf_setup.md b/kubevirt-patch/emt_setup.md similarity index 98% rename from kubevirt-patch/tiber_mf_setup.md rename to kubevirt-patch/emt_setup.md index 6f63a2a..9373270 100644 --- a/kubevirt-patch/tiber_mf_setup.md +++ b/kubevirt-patch/emt_setup.md @@ -61,9 +61,9 @@ crw-rw-r--. 1 root qemu 189, 8 May 15 22:14 009 ... ``` -# 3. Display setup for TiberOS +# 3. Display setup for EMT OS -TiberOS boots with no GUI and prompts for user login, login using default credentials\ +EMT OS boots with no GUI and prompts for user login, login using default credentials\ XSERVER is installed by default, make the below settings before starting X server ## 3.1 Disable DPMS and screen blanking on the X Window System diff --git a/kubevirt-patch/kubevirt-cr-gfx-sriov.yaml b/kubevirt-patch/kubevirt-cr-gfx-sriov.yaml new file mode 100644 index 0000000..1a83321 --- /dev/null +++ b/kubevirt-patch/kubevirt-cr-gfx-sriov.yaml @@ -0,0 +1,32 @@ +apiVersion: kubevirt.io/v1 +kind: KubeVirt +metadata: + name: kubevirt + namespace: kubevirt +spec: + configuration: + developerConfiguration: + featureGates: + - GPU + - HostDevices + - Sidecar + permittedHostDevices: + pciHostDevices: + - pciVendorSelector: "8086:4680" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + - pciVendorSelector: "8086:46b3" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + - pciVendorSelector: "8086:46a6" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + - pciVendorSelector: "8086:56c0" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + - pciVendorSelector: "8086:56c1" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + - pciVendorSelector: "8086:a7a0" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false \ No newline at end of file diff --git a/kubevirt-patch/kubevirt-offline-install.md b/kubevirt-patch/kubevirt-offline-install.md index 547ba9f..5824533 100644 --- a/kubevirt-patch/kubevirt-offline-install.md +++ b/kubevirt-patch/kubevirt-offline-install.md @@ -1,11 +1,11 @@ # Kubevirt installation using TAR files -This version of Kubevirt is built on release tag v1.5.0 along with GTK library support for enabling Display Virtualization and Intel Graphics SR-IOV patched QEMU version 8.2.1 that supports local display of edge node. Hence tagged the version as v1.5.0_DV and is shared as a [Kubevirt TAR](link_to_kubevier_tar) +This version of Kubevirt is built on release tag v1.5.0 along with GTK library support for enabling Display Virtualization and Intel Graphics SR-IOV patched QEMU version 8.2.1 that supports local display of edge node. Hence tagged the version as v1.5.0_DV and is shared as a [Kubevirt TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-kubevirt-v0.1.tar.gz) -Also the Device-Plugin has been shared as a [Device-Plugin TAR](link_to_dp_tar) to support enabling Display Virtualization on local display of edge node +Also the Device-Plugin has been shared as a [Device-Plugin TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-device-plugin-v0.1.tar.gz) to support enabling Display Virtualization on local display of edge node ## Steps 1. Ensure Kubernetes is installed and local cluster is running. -2. Download [Kubevirt TAR](link_to_kubevier_tar) and [Device-Plugin TAR](link_to_dp_tar) to the host system +2. Download [Kubevirt TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-kubevirt-v0.1.tar.gz) and [Device-Plugin TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-device-plugin-v0.1.tar.gz) to the host system 3. Extract TAR files ```sh mkdir -p ~/display-virtualization @@ -27,6 +27,19 @@ Also the Device-Plugin has been shared as a [Device-Plugin TAR](link_to_dp_tar) sudo ctr -n k8s.io images import device-plugin.tar sudo ctr -n k8s.io images import busybox.tar ``` + + Alternatively + ```sh + sudo k3s ctr i import virt-operator.tar + sudo k3s ctr i import virt-api.tar + sudo k3s ctr i import virt-controller.tar + sudo k3s ctr i import virt-handler.tar + sudo k3s ctr i import virt-launcher.tar + sudo k3s ctr i import sidecar-shim.tar + + sudo k3s ctr i import device-plugin.tar + sudo k3s ctr i import busybox.tar + ``` 5. Verify the images are imported correctly ```sh sudo crictl images | grep localhost @@ -68,3 +81,50 @@ Also the Device-Plugin has been shared as a [Device-Plugin TAR](link_to_dp_tar) . . ``` +8. Enable Virt-Handler to discover Graphics VFs + Update KubeVirt custom resource configuration to enable virt-handler to discover graphics VFs on the host. All discovered VFs will be published as *allocatable* resource + + **Update Graphics Device ID in `kubevirt-cr-gfx-sriov.yaml` if not found** + - Read the Device ID of Intel Graphics Card from Host, Ex: for RPL + ```sh + $ cat /sys/devices/pci0000\:00/0000\:00\:02.0/device + + 0xa7a0 + ``` + - Add the Device ID in `pciHostDevices` section + ```yaml + - pciVendorSelector: "8086:a7a0" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + ``` + + Apply the YAML changes + ```sh + kubectl apply -f manifests/kubevirt-cr-gfx-sriov.yaml + ``` + + **Check for presence of `intel.com/sriov-gpudevices` resource** + + ```sh + kubectl describe nodes + ``` + Output: + ```sh + Capacity: + intel.com/sriov-gpudevice: 7 + Allocatable: + intel.com/sriov-gpudevice: 7 + Allocated resources: + Resource Requests Limits + -------- -------- ------ + intel.com/sriov-gpudevice 0 0 + ``` + > [!Note] + > Please wait for all virt-handler pods to complete restarts\ + > The value of **Requests** and **Limits** will increase upon successful resource allocation to running pods/VMs + +9. Install CDI + ```sh + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-operator.yaml + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-cr.yaml + ``` diff --git a/kubevirt-patch/rpmbuild-install.md b/kubevirt-patch/rpmbuild-install.md index 34f8338..1f52c13 100644 --- a/kubevirt-patch/rpmbuild-install.md +++ b/kubevirt-patch/rpmbuild-install.md @@ -5,7 +5,7 @@ System should be installed with rpm build environment. ## Build -1. Download and copy the [Kubevirt TAR](link_to_kubevier_tar) and [Device-Plugin TAR](link_to_dp_tar) to `SOURCES` +1. Download and copy the [Kubevirt TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-kubevirt-v0.1.tar.gz) and [Device-Plugin TAR](https://github.com/open-edge-platform/edge-desktop-virtualization/releases/download/pre-release-v0.1/intel-idv-device-plugin-v0.1.tar.gz) to `SOURCES` 2. Copy the desktop-virtualization-k3s.spec to `SPECS` 3. Build the RPM ```sh @@ -52,6 +52,18 @@ System should be installed with rpm build environment. sudo ctr -n k8s.io images import device-plugin.tar sudo ctr -n k8s.io images import busybox.tar ``` + Alternatively + ```sh + sudo k3s ctr i import virt-operator.tar + sudo k3s ctr i import virt-api.tar + sudo k3s ctr i import virt-controller.tar + sudo k3s ctr i import virt-handler.tar + sudo k3s ctr i import virt-launcher.tar + sudo k3s ctr i import sidecar-shim.tar + + sudo k3s ctr i import device-plugin.tar + sudo k3s ctr i import busybox.tar + ``` 5. Verify the images are imported correctly ```sh sudo crictl images | grep localhost @@ -92,4 +104,51 @@ System should be installed with rpm build environment. kubevirt kubevirt.kubevirt.io/kubevirt 9d Deployed . . - ``` \ No newline at end of file + ``` +8. Enable Virt-Handler to discover Graphics VFs + Update KubeVirt custom resource configuration to enable virt-handler to discover graphics VFs on the host. All discovered VFs will be published as *allocatable* resource + + **Update Graphics Device ID in `kubevirt-cr-gfx-sriov.yaml` if not found** + - Read the Device ID of Intel Graphics Card from Host, Ex: for RPL + ```sh + $ cat /sys/devices/pci0000\:00/0000\:00\:02.0/device + + 0xa7a0 + ``` + - Add the Device ID in `pciHostDevices` section + ```yaml + - pciVendorSelector: "8086:a7a0" + resourceName: "intel.com/sriov-gpudevice" + externalResourceProvider: false + ``` + + Apply the YAML changes + ```sh + kubectl apply -f manifests/kubevirt-cr-gfx-sriov.yaml + ``` + + **Check for presence of `intel.com/sriov-gpudevices` resource** + + ```sh + kubectl describe nodes + ``` + Output: + ```sh + Capacity: + intel.com/sriov-gpudevice: 7 + Allocatable: + intel.com/sriov-gpudevice: 7 + Allocated resources: + Resource Requests Limits + -------- -------- ------ + intel.com/sriov-gpudevice 0 0 + ``` + > [!Note] + > Please wait for all virt-handler pods to complete restarts\ + > The value of **Requests** and **Limits** will increase upon successful resource allocation to running pods/VMs + +9. Install CDI + ```sh + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-operator.yaml + kubectl apply -f https://github.com/kubevirt/containerized-data-importer/releases/download/v1.60.3/cdi-cr.yaml + ``` diff --git a/sample-application/discrete/README.md b/sample-application/discrete/README.md index 46157c2..9232e02 100644 --- a/sample-application/discrete/README.md +++ b/sample-application/discrete/README.md @@ -96,7 +96,6 @@ Ex. for `vm1` the image name in CDI is `vm1-win11-image` ``` - Upload image, use **.qcow2** or **.img** ```sh - cd tiber/kubevirt/cmd/virtctl/ ./virtctl image-upload --uploadproxy-url=https://10.43.51.68 --insecure dv vm1-win11-image --size=100Gi --access-mode=ReadWriteOnce --force-bind --image-path=/home/guest/disk.qcow2 --force-bind ``` diff --git a/sample-application/single/README.md b/sample-application/single/README.md index ad12a42..473d5d0 100644 --- a/sample-application/single/README.md +++ b/sample-application/single/README.md @@ -82,7 +82,6 @@ Ex. for `vm1` the image name(dataVolumeName) in CDI is `vm1-win11-image` ``` - Upload image, use **.qcow2** or **.img** ```sh - cd tiber/kubevirt/cmd/virtctl/ ./virtctl image-upload --uploadproxy-url=https://10.43.51.68 --insecure dv vm1-win11-image --size=100Gi --access-mode=ReadWriteOnce --force-bind --image-path=/home/guest/disk.qcow2 --force-bind ```