Skip to content

Commit 3854b19

Browse files
sjhala-ccssjhala-ccs
authored
Merge pull request #75714 from orelmisan/real-time-guest-config
CNV-40785: Configure guest for real-time
2 parents b006574 + fbf5820 commit 3854b19

File tree

1 file changed

+103
-20
lines changed

1 file changed

+103
-20
lines changed

‎modules/virt-configuring-vm-real-time.adoc

Lines changed: 103 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
[id="virt-configuring-vm-real-time_{context}"]
77
= Configuring a virtual machine for real-time workloads
88

9-
You can configure a virtual machines (VM) to run real-time workloads.
9+
You can configure a virtual machine (VM) to run real-time workloads.
1010

1111
.Prerequisites
1212
* Your cluster is configured to run real-time workloads.
@@ -23,12 +23,13 @@ kind: VirtualMachine
2323
metadata:
2424
name: realtime-vm
2525
spec:
26+
running: true
2627
template:
2728
metadata:
2829
annotations:
29-
cpu-load-balancing.crio.io: disable <1>
30-
cpu-quota.crio.io: disable <2>
31-
irq-load-balancing.crio.io: disable <3>
30+
cpu-load-balancing.crio.io: disable # <1>
31+
cpu-quota.crio.io: disable # <2>
32+
irq-load-balancing.crio.io: disable # <3>
3233
spec:
3334
domain:
3435
cpu:
@@ -38,8 +39,8 @@ spec:
3839
numa:
3940
guestMappingPassthrough: {}
4041
realtime: {}
41-
sockets: 1 <4>
42-
cores: 4 <5>
42+
sockets: 1 # <4>
43+
cores: 4 # <5>
4344
threads: 1
4445
devices:
4546
autoattachGraphicsDevice: false
@@ -49,56 +50,138 @@ spec:
4950
memory:
5051
guest: 4Gi
5152
hugepages:
52-
pageSize: 1Gi <6>
53+
pageSize: 1Gi # <6>
5354
terminationGracePeriodSeconds: 0
5455
# ...
5556
----
5657
<1> This annotation specifies that load balancing is disabled for CPUs that are used by the container.
5758
<2> This annotation specifies that the CPU quota is disabled for CPUs that are used by the container.
5859
<3> This annotation specifies that interrupt request (IRQ) load balancing is disabled for CPUs that are used by the container.
59-
<4> The number of sockets inside the VM.
60+
<4> The number of sockets inside the VM.
6061
<5> The number of cores inside the VM. This must be a value greater than or equal to `1`.
6162
<6> The size of the huge pages. The possible values for x86-64 architectures are `1Gi` and `2Mi`. In this example, the request is for 4 huge pages of size 1 Gi.
62-
6363
. Apply the `VirtualMachine` manifest:
6464
+
6565
[source,terminal]
6666
----
6767
$ oc apply -f <file_name>.yaml
6868
----
69-
70-
. Configure the guest operating system. The following example shows the configuration steps for a {op-system-base} 8 operating system:
69+
. Configure the guest operating system. The following example shows the configuration steps for a {op-system-base} 9 operating system:
7170
.. Run the following command to connect to the VM console:
7271
+
7372
[source,terminal]
7473
----
7574
$ virtctl console <vm_name>
7675
----
77-
78-
.. Configure huge pages by using the GRUB boot loader command-line interface. In the following example, 8 1G huge pages are specified.
76+
.. If you are not already logged in as a root user, switch to the root user account to execute the remaining configuration steps.
77+
.. Disable the `irqbalance` service by using the following command:
7978
+
8079
[source,terminal]
8180
----
82-
$ grubby --update-kernel=ALL --args="default_hugepagesz=1GB hugepagesz=1G hugepages=8"
81+
# systemctl disable irqbalance && systemctl stop irqbalance
8382
----
84-
85-
.. To achieve low-latency tuning by using the `cpu-partitioning` profile in the TuneD application, run the following commands:
83+
.. Enable the real-time and network function virtualization (NFV) repositories:
8684
+
8785
[source,terminal]
8886
----
89-
$ dnf install -y tuned-profiles-cpu-partitioning
87+
# subscription-manager repos --enable rhel-9-for-x86_64-rt-rpms --enable rhel-9-for-x86_64-nfv-rpms
9088
----
89+
.. Install the necessary packages by running the following command:
9190
+
9291
[source,terminal]
9392
----
94-
$ echo isolated_cores=2-9 > /etc/tuned/cpu-partitioning-variables.conf
93+
# dnf install tuned cloud-init
9594
----
96-
The first two CPUs (0 and 1) are set aside for house keeping tasks and the rest are isolated for the real-time application.
95+
.. To achieve low-latency tuning by using the `realtime-virtual-guest` profile in the TuneD application, run the following commands:
9796
+
9897
[source,terminal]
9998
----
100-
$ tuned-adm profile cpu-partitioning
99+
# dnf install kernel-rt realtime-tests tuned-profiles-realtime
101100
----
101+
+
102+
[source,terminal]
103+
----
104+
# dnf install tuned-profiles-nfv-guest
105+
----
106+
.. Edit the `/etc/tuned/realtime-virtual-guest-variables.conf` file:
107+
+
108+
[source,conf]
109+
----
110+
#
111+
# Variable settings below override the definitions from the
112+
# /etc/tuned/realtime-variables.conf file.
113+
114+
#
115+
# Core isolation
116+
#
117+
# The 'isolated_cores=' variable below controls which cores should be
118+
# isolated. By default we reserve 1 core per socket for housekeeping
119+
# and isolate the rest. But you can isolate any range as shown in the
120+
# examples below. Just remember to keep only one isolated_cores= line.
121+
#
122+
# Examples:
123+
# isolated_cores=2,4-7
124+
# isolated_cores=2-23
125+
#
126+
# Reserve 1 core per socket for housekeeping, isolate the rest.
127+
# Change this for a core list or range as shown above.
128+
isolated_cores=2-3 <1>
102129
130+
#
131+
# Uncomment the 'isolate_managed_irq=Y' bellow if you want to move kernel
132+
# managed IRQs out of isolated cores. Note that this requires kernel
133+
# support. Please only specify this parameter if you are sure that the
134+
# kernel supports it.
135+
#
136+
isolate_managed_irq=Y <2>
137+
138+
#
139+
# Set the desired combined queue count value using the parameter provided
140+
# below. Ideally this should be set to the number of housekeeping CPUs i.e.,
141+
# in the example given below it is assumed that the system has 4 housekeeping
142+
# (non-isolated) CPUs.
143+
#
144+
# netdev_queue_count=4
145+
----
146+
<1> The first two CPUs (0 and 1) are set aside for house keeping tasks and the rest are isolated for the real-time application.
147+
<2> Set the `isolate_managed_irq` parameter to `Y` to move kernel-managed interrupt requests out of isolated cores.
148+
.. Activate the TuneD profile:
149+
+
150+
[source,terminal]
151+
----
152+
# tuned-adm profile realtime-virtual-guest
153+
----
154+
.. Set the real-time kernel as the default by using the GRUB boot loader command-line interface:
155+
+
156+
[source,terminal]
157+
----
158+
# grubby --set-default=/boot/vmlinuz-<installed_rt_kernel_version>
159+
----
160+
.. Set the kernel arguments by using the GRUB boot loader command-line interface:
161+
+
162+
[source,terminal]
163+
----
164+
# grubby --args="iommu=pt intel_iommu=on default_hugepagesz=1G idle=poll" --update-kernel=$(grubby --default-kernel)
165+
----
103166
. Restart the VM to apply the changes.
104167

168+
.Verification
169+
* Use the `cyclictest` tool to verify that the real-time guest is configured properly:
170+
+
171+
[source,terminal]
172+
----
173+
# cyclictest --priority 1 --policy fifo -h 50 -a 2-3 --mainaffinity 0,1 -t 2 -m -q -i 200 -D 12h
174+
----
175+
where:
176+
`-a`:: Specifies the CPU set on which the test runs. This is the same as the isolated CPUs that you configured in the `realtime-variables.conf` file.
177+
`-D`:: Specifies the test duration. Append `m`, `h`, or `d` to specify minutes, hours or days.
178+
179+
+
180+
.Example output
181+
[source,terminal]
182+
----
183+
# Min Latencies: 00004 00004
184+
# Avg Latencies: 00004 00004
185+
# Max Latencies: 00014 00013 <1>
186+
----
187+
<1> The `Max Latencies` value in the output must be less than 40 micro seconds.

0 commit comments

Comments
 (0)