Skip to content

Commit ec96f92

Browse files
authored
Merge pull request #84389 from MirzWeiss/CNV-13372
CNV-13372: Add post-copy live migration information to docs.
2 parents 3bafc1d + 50dc15b commit ec96f92

5 files changed

+93
-3
lines changed
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
2+
// Module included in the following assemblies:
3+
//
4+
// * virt/live_migration/virt-configuring-live-migration.adoc
5+
6+
:_mod-docs-content-type: PROCEDURE
7+
[id="virt-configuring-live-migration-heavy_{context}"]
8+
= Configure live migration for heavy workloads
9+
10+
When migrating a VM running a heavy workload (for example, database processing) with higher memory dirty rates, you need a higher bandwidth to complete the migration.
11+
12+
If the dirty rate is too high, the migration from one node to another does not converge. To prevent this, enable post copy mode.
13+
14+
Post copy mode triggers if the initial pre-copy phase does not complete within the defined timeout. During post copy, the VM CPUs pause on the source host while transferring the minimum required memory pages. Then the VM CPUs activate on the destination host, and the remaining memory pages transfer into the destination node at runtime.
15+
16+
Configure live migration for heavy workloads by updating the `HyperConverged` custom resource (CR), which is located in the `{CNVNamespace}` namespace.
17+
18+
.Procedure
19+
20+
. Edit the `HyperConverged` CR and add the necessary parameters for migrating heavy workloads:
21+
+
22+
[source,terminal,subs="attributes+"]
23+
----
24+
$ oc edit hyperconverged kubevirt-hyperconverged -n {CNVNamespace}
25+
----
26+
+
27+
.Example configuration file
28+
[source,yaml,subs="attributes+"]
29+
----
30+
apiVersion: hco.kubevirt.io/v1beta1
31+
kind: HyperConverged
32+
metadata:
33+
name: kubevirt-hyperconverged
34+
namespace: {CNVNamespace}
35+
spec:
36+
liveMigrationConfig:
37+
bandwidthPerMigration: 0Mi <1>
38+
completionTimeoutPerGiB: 150 <2>
39+
parallelMigrationsPerCluster: 5 <3>
40+
parallelOutboundMigrationsPerNode: 1 <4>
41+
progressTimeout: 150 <5>
42+
allowPostCopy: true <6>
43+
----
44+
<1> Bandwidth limit of each migration, where the value is the quantity of bytes per second. The default is `0`, which is unlimited.
45+
<2> The migration is canceled if it is not completed in this time, and triggers post copy mode, when post copy is enabled. This value is measured in seconds per GiB of memory. You can lower `completionTimeoutPerGiB` to trigger post copy mode earlier in the migration process, or raise the `completionTimeoutPerGiB` to trigger post copy mode later in the migration process.
46+
<3> Number of migrations running in parallel in the cluster. The default is `5`. Keeping the `parallelMigrationsPerCluster` setting low is better when migrating heavy workloads.
47+
<4> Maximum number of outbound migrations per node. Configure a single VM per node for heavy workloads.
48+
<5> The migration is canceled if memory copy fails to make progress in this time. This value is measured in seconds. Increase this parameter for large memory sizes running heavy workloads.
49+
<6> Use post copy mode when memory dirty rates are high to ensure the migration converges. Set `allowPostCopy` to `true` to enable post copy mode.
50+
51+
. Optional: If your main network is too busy for the migration, configure a secondary, dedicated migration network.
52+
53+
[NOTE]
54+
====
55+
Post copy mode can impact performance during the transfer, and should not be used for critical data, or with unstable networks.
56+
====

modules/virt-configuring-live-migration-limits.adoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
= Configuring live migration limits and timeouts
99

1010
Configure live migration limits and timeouts for the cluster by updating the `HyperConverged` custom resource (CR), which is located in the
11-
`openshift-cnv` namespace.
11+
`{CNVNamespace}` namespace.
1212

1313
.Procedure
1414

@@ -34,12 +34,14 @@ spec:
3434
parallelMigrationsPerCluster: 5 <3>
3535
parallelOutboundMigrationsPerNode: 2 <4>
3636
progressTimeout: 150 <5>
37+
allowPostCopy: false <6>
3738
----
3839
<1> Bandwidth limit of each migration, where the value is the quantity of bytes per second. For example, a value of `2048Mi` means 2048 MiB/s. Default: `0`, which is unlimited.
3940
<2> The migration is canceled if it has not completed in this time, in seconds per GiB of memory. For example, a VM with 6GiB memory times out if it has not completed migration in 4800 seconds. If the `Migration Method` is `BlockMigration`, the size of the migrating disks is included in the calculation.
4041
<3> Number of migrations running in parallel in the cluster. Default: `5`.
4142
<4> Maximum number of outbound migrations per node. Default: `2`.
4243
<5> The migration is canceled if memory copy fails to make progress in this time, in seconds. Default: `150`.
44+
<6> If a VM is running a heavy workload and the memory dirty rate is too high, this can prevent the migration from one node to another from converging. To prevent this, you can enable post copy mode. By default, `allowPostCopy` is set to `false`.
4345
4446
[NOTE]
4547
====

modules/virt-vm-migration-tuning.adoc

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
2+
// Module included in the following assemblies:
3+
//
4+
// * virt/live_migration/virt-about-live-migration.adoc
5+
6+
:_mod-docs-content-type: CONCEPT
7+
[id="virt-vm-migration-tuning_{context}"]
8+
= VM migration tuning
9+
10+
You can adjust your cluster-wide live migration settings based on the type of workload and migration scenario. This enables you to control how many VMs migrate at the same time, the network bandwidth you want to use for each migration, and how long {VirtProductName} attempts to complete the migration before canceling the process. Configure these settings in the `HyperConverged` custom resource (CR).
11+
12+
If you are migrating multiple VMs per node at the same time, set a `bandwidthPerMigration` limit to prevent a large or busy VM from using a large portion of the node’s network bandwidth. By default, the `bandwidthPerMigration` value is `0`, which means unlimited.
13+
14+
A large VM running a heavy workload (for example, database processing), with higher memory dirty rates, requires a higher bandwidth to complete the migration.
15+
16+
[NOTE]
17+
====
18+
Post copy mode, when enabled, triggers if the initial pre-copy phase does not complete within the defined timeout. During post copy, the VM CPUs pause on the source host while transferring the minimum required memory pages. Then the VM CPUs activate on the destination host, and the remaining memory pages transfer into the destination node at runtime. This can impact performance during the transfer.
19+
20+
Post copy mode should not be used for critical data, or with unstable networks.
21+
====

virt/live_migration/virt-about-live-migration.adoc

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@ include::_attributes/common-attributes.adoc[]
66

77
toc::[]
88

9-
Live migration is the process of moving a running virtual machine (VM) to another node in the cluster without interrupting the virtual workload. By default, live migration traffic is encrypted using Transport Layer Security (TLS).
9+
Live migration is the process of moving a running virtual machine (VM) to another node in the cluster without interrupting the virtual workload. Live migration enables smooth transitions during cluster upgrades or any time a node needs to be drained for maintenance or configuration changes.
10+
11+
By default, live migration traffic is encrypted using Transport Layer Security (TLS).
1012

1113
[id="live-migration-requirements_virt-about-live-migration"]
1214
== Live migration requirements
@@ -30,12 +32,15 @@ The default number of migrations that can run in parallel in the cluster is 5.
3032
* If a VM uses a host model CPU, the nodes must support the CPU.
3133
* xref:../../virt/vm_networking/virt-dedicated-network-live-migration.adoc#virt-dedicated-network-live-migration[Configuring a dedicated Multus network] for live migration is highly recommended. A dedicated network minimizes the effects of network saturation on tenant workloads during migration.
3234

35+
include::modules/virt-vm-migration-tuning.adoc[leveloffset=+1]
36+
3337
[id="common-live-migration-tasks_virt-about-live-migration"]
3438
== Common live migration tasks
3539

3640
You can perform the following live migration tasks:
3741

3842
* xref:../../virt/live_migration/virt-configuring-live-migration.adoc#virt-configuring-live-migration[Configure live migration settings]
43+
* xref:../../virt/live_migration/virt-configuring-live-migration.adoc#virt-configuring-live-migration-heavy_virt-configuring-live-migration[Configure live migration for heavy workloads]
3944
* xref:../../virt/live_migration/virt-initiating-live-migration.adoc#virt-initiating-live-migration[Initiate and cancel live migration]
4045
* Monitor the progress of all live migrations in the *Migration* tab of the {virtproductname} web console.
4146
* View VM migration metrics in the *Metrics* tab of the web console.
@@ -44,6 +49,5 @@ You can perform the following live migration tasks:
4449
[id="additional-resources_virt-about-live-migration"]
4550
== Additional resources
4651
* xref:../../virt/monitoring/virt-prometheus-queries.adoc#virt-live-migration-metrics_virt-prometheus-queries[Prometheus queries for live migration]
47-
* link:https://access.redhat.com/articles/6994974#vm-migration-tuning[VM migration tuning]
4852
* xref:../../virt/nodes/virt-node-maintenance.adoc#run-strategies[VM run strategies]
4953
* xref:../../virt/nodes/virt-node-maintenance.adoc#eviction-strategies[VM and cluster eviction strategies]

virt/live_migration/virt-configuring-live-migration.adoc

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,13 @@ You can configure live migration policies to apply different migration configura
1212

1313
include::modules/virt-configuring-live-migration-limits.adoc[leveloffset=+1]
1414

15+
include::modules/virt-configuring-live-migration-heavy.adoc[leveloffset=+1]
16+
17+
[role="_additional-resources"]
18+
[id="additional-resources_virt-configuring-live-migration-heavy"]
19+
== Additional resources
20+
* xref:../../virt/vm_networking/virt-dedicated-network-live-migration.adoc#virt-configuring-secondary-network-vm-live-migration_virt-dedicated-network-live-migration[Configuring a dedicated network for live migration]
21+
1522
[id="live-migration-policies"]
1623
== Live migration policies
1724

0 commit comments

Comments
 (0)