Merge pull request #79038 from laubai/osdocs-11160-maxsurge-maxunavailable

OSDOCS#11160: New max-unavailable and max-surge parameters

Author: kcarmichael08

modules/rosa-create-objects.adoc

@@ -744,6 +744,21 @@ a|--kubelet-configs <kubeletconfig_name>
 |--min-replicas
 |Specifies the minimum number of compute nodes when enabling autoscaling.
 
+//OSDOCS-11160: HCP only, but need to wait on separate HCP publishing
+//ifdef::openshift-rosa-hcp[]
+|--max-surge
+a| For {hcp-title-first} clusters, the `max-surge` parameter defines the number of new nodes that can be provisioned in excess of the desired number of replicas for the machine pool, as configured using the `--replicas` parameter, or as determined by the autoscaler when autoscaling is enabled. This can be an absolute number (for example, `2`) or a percentage of the machine pool size (for example, `20%`), but must use the same unit as the `max-unavailable` parameter.
+
+The default value is `1`, meaning that the maximum number of nodes in the machine pool during an upgrade is 1 plus the desired number of replicas for the machine pool. In this situation, one excess node can be provisioned before existing nodes need to be made unavailable. The number of nodes that can be provisioned simultaneously during an upgrade is `max-surge` plus `max-unavailable`.
+
+|--max-unavailable
+a|For {hcp-title-first} clusters, the `max-unavailable` parameter defines the number of nodes that can be made unavailable in a machine pool during an upgrade, before new nodes are provisioned. This can be an absolute number (for example, `2`) or a percentage of the current replica count in the machine pool (for example, `20%`), but must use the same unit as the `max-surge` parameter.
+
+The default value is `0`, meaning that no outdated nodes are removed before new nodes are provisioned. The valid range for this value is from `0` to the current machine pool size, or from `0%` to `100%`. The total number of nodes that can be upgraded simultaneously during an upgrade is `max-surge` plus `max-unavailable`.
+
+//endif::openshift-rosa-hcp[]
+// end OSDOCS-11160: HCP only, when separate docs are available
+
 |--name
 |Required: The name (string) for the machine pool.
 
@@ -797,6 +812,16 @@ Add a machine pool that is named `mp-1` with 3 replicas of `m5.xlarge` to a clus
 $ rosa create machinepool --cluster=mycluster --replicas=3 --instance-type=m5.xlarge --name=mp-1
 ----
 
+Add a machine pool (`mp-1`) to a {hcp-title-first} cluster, configuring 6 replicas and the following upgrade behavior:
+
+* Allow up to 2 excess nodes to be provisioned during an upgrade.
+* Ensure that no more than 3 nodes are unavailable during an upgrade.
+
+[source,terminal]
+----
+$ rosa create machinepool --cluster=mycluster --replicas=6 --name=mp-1 --max-surge=2 --max-unavailable=3
+----
+
 Add a machine pool with labels to a cluster.
 
 [source,terminal]

modules/rosa-edit-objects.adoc

@@ -234,7 +234,7 @@ Allows edits to the machine pool in a cluster.
 .Syntax
 [source,terminal]
 ----
-$ rosa edit machinepool --cluster=<cluster_name> | <cluster_id> <machinepool_ID> [arguments]
+$ rosa edit machinepool --cluster=<cluster_name_or_id> <machinepool_ID> [arguments]
 ----
 
 .Arguments
@@ -263,6 +263,20 @@ a|--kubelet-configs <kubeletconfig_name>
 |--min-replicas
 |Specifies the minimum number of compute nodes when enabling autoscaling.
 
+//OSDOCS-11160: HCP only, but need to wait on separate HCP publishing
+//ifdef::openshift-rosa-hcp[]
+|--max-surge
+a| For {hcp-title-first} clusters, the `max-surge` parameter defines the number of new nodes that can be provisioned in excess of the desired number of replicas for the machine pool, as configured using the `--replicas` parameter, or as determined by the autoscaler when autoscaling is enabled. This can be an absolute number (for example, `2`) or a percentage of the machine pool size (for example, `20%`), but must use the same unit as the `max-unavailable` parameter.
+
+The default value is `1`, meaning that the maximum number of nodes in the machine pool during an upgrade is 1 plus the desired number of replicas for the machine pool. In this situation, one excess node can be provisioned before existing nodes need to be made unavailable. The number of nodes that can be provisioned simultaneously during an upgrade is `max-surge` plus `max-unavailable`.
+
+|--max-unavailable
+a|For {hcp-title-first} clusters, the `max-unavailable` parameter defines the number of nodes that can be made unavailable in a machine pool during an upgrade, before new nodes are provisioned. This can be an absolute number (for example, `2`) or a percentage of the current replica count in the machine pool (for example, `20%`), but must use the same unit as the `max-surge` parameter.
+
+The default value is `0`, meaning that no outdated nodes are removed before new nodes are provisioned. The valid range for this value is from `0` to the current machine pool size, or from `0%` to `100%`. The total number of nodes that can be upgraded simultaneously during an upgrade is `max-surge` plus `max-unavailable`.
+//endif::openshift-rosa-hcp[]
+// end OSDOCS-11160: HCP only, when separate docs are available
+
 |--node-drain-grace-period
 |Specifies the node drain grace period when upgrading or replacing the machine pool. (This is for {hcp-title} clusters only.)
 
@@ -297,33 +311,43 @@ Set 4 replicas on a machine pool named `mp1` on a cluster named `mycluster`.
 
 [source,terminal]
 ----
-$ rosa edit machinepool --cluster=mycluster --replicas=4 --name=mp1
+$ rosa edit machinepool --cluster=mycluster --replicas=4 mp1
 ----
 
 Enable autoscaling on a machine pool named `mp1` on a cluster named `mycluster`.
 
 [source,terminal]
 ----
-$ rosa edit machinepool --cluster=mycluster --enable-autoscaling --min-replicas=3 --max-replicas=5 --name=mp1
+$ rosa edit machinepool --cluster=mycluster --enable-autoscaling --min-replicas=3 --max-replicas=5 mp1
 ----
 
 Disable autoscaling on a machine pool named `mp1` on a cluster named `mycluster`.
 
 [source,terminal]
 ----
-$ rosa edit machinepool --cluster=mycluster  --enable-autoscaling=false --replicas=3 --name=mp1
+$ rosa edit machinepool --cluster=mycluster  --enable-autoscaling=false --replicas=3 mp1
 ----
 
 Modify the autoscaling range on a machine pool named `mp1` on a cluster named `mycluster`.
 
 [source,terminal]
 ----
-$ rosa edit machinepool --max-replicas=9 --cluster=mycluster --name=mp1
+$ rosa edit machinepool --max-replicas=9 --cluster=mycluster mp1
+----
+
+On {hcp-title-first} clusters, edit the `mp1` machine pool to add the following behavior during upgrades:
+
+* Allow up to 2 excess nodes to be provisioned during an upgrade.
+* Ensure that no more than 3 nodes are unavailable during an upgrade.
+
+[source,terminal]
+----
+$ rosa edit machinepool --cluster=mycluster mp1 --max-surge=2 --max-unavailable=3
 ----
 
-Associate a `KubeletConfig` object with an existing machine pool on a {hcp-title-first} cluster.
+Associate a `KubeletConfig` object with an existing `high-pid-pool` machine pool on a {hcp-title} cluster.
 
 [source,terminal]
 ----
-$ rosa edit machinepool -c mycluster --kubelet-configs=set-high-pids --name high-pid-pool
+$ rosa edit machinepool -c mycluster --kubelet-configs=set-high-pids high-pid-pool
 ----

modules/rosa-hcp-upgrade-options.adoc

@@ -0,0 +1,34 @@
+:_mod-docs-content-type: CONCEPT
+[id="rosa-upgrade-options_{context}"]
+= Upgrade options for {hcp-title} clusters
+
+In OpenShift, upgrading means provisioning a new component with updated software and using it to replace an existing component that has outdated software.
+
+You can control the impact of upgrades to your workload by controlling which parts of the cluster are upgraded, for example:
+
+Upgrade only the hosted control plane:: This does not impact your worker nodes.
+
+Upgrade nodes in a single machine pool:: This initiates a rolling replacement of nodes in the specified machine pool, and temporarily impacts the worker nodes on that machine pool. This does not impact nodes on other machine pools in the cluster.
+
+Upgrade nodes in multiple machine pools simultaneously:: This initiates a rolling replacement of nodes in the specified machine pools, and temporarily impacts the worker nodes on those machine pools. You can run this type of upgrade as a single command, or as multiple commands.
+
+Upgrade the whole cluster in sequence:: This initiates upgrade of the hosted control plane, followed by a rolling replacement of nodes in the specified machine pools. These upgrades occur in sequence because the hosted control plane and the machine pools cannot be upgraded at the same time. When an upgrade of the hosted control plane is in progress, nodes in the machine pools cannot be upgraded. When upgrade is in progress for nodes in the machine pools, the hosted control plane cannot be upgraded.
++
+[IMPORTANT]
+====
+To maintain compatibility between nodes in the cluster, nodes in machine pools cannot use a newer version than the hosted control plane. This means that the hosted control plane should always be upgraded to a given version before any machine pools are upgraded to the same version.
+====
+
+The time required to upgrade the hosted control plane varies depending on your workload configuration.
+
+The time required to upgrade a machine pool varies according to the number of worker nodes in the machine pool (`--replicas` or `--max-replicas`).
+
+You can further control the time required for an upgrade, and the impact of an upgrade to your workload, by editing the `--max-surge` and `--max-unavailable` values for each machine pool. These options control the number of nodes that can be upgraded simultaneously, and whether an upgrade provisions excess nodes or makes some existing nodes unavailable or both, for example:
+
+* **To prioritize high workload availability**, you can provision excess nodes instead of making existing nodes unavailable by setting a higher value for `--max-surge` and setting `--max-unavailable` to `0`.
+* **To prioritize lower infrastructure costs**, you can make some existing nodes unavailable and avoid provisioning excess nodes by setting a higher value for `--max-unavailable` and setting `--max-surge` to `0`.
+* **To prioritize upgrade speed by upgrading multiple nodes simultaneously**, you can provision excess nodes and allow some existing nodes to be made unavailable by configuring moderate values for both `--max-surge` and `--max-unavailable`.
+
+For more information about these parameters and their usage, see the _ROSA CLI reference_ for `rosa edit machinepool`.
+
+//Additional resources included in assembly.

modules/rosa-hcp-upgrading-cli-control-plane.adoc

@@ -0,0 +1,94 @@
+// Module included in the following assemblies:
+//
+// * upgrading/rosa-hcp-upgrading.adoc
+
+// NOTE: This module is included several times in the same upgrade assembly.
+
+:_mod-docs-content-type: PROCEDURE
+[id="rosa-hcp-upgrading-cli-control-plane_{context}"]
+// HCP-ONLY: Conditions for upgrading the hosted control plane WITHOUT upgrading any machine pools
+ifeval::["{context}" != "rosa-hcp-upgrading-whole-cluster"]
+= Upgrading the hosted control plane with the ROSA CLI
+
+You can manually upgrade the hosted control plane of a {hcp-title} cluster by using the ROSA CLI. This method schedules the control plane for an upgrade if a more recent version is available, either immediately, or at a specified future time.
+
+[NOTE]
+====
+Your control plane only supports machine pools within two minor Y-stream versions. For example, a {hcp-title} cluster with a control plane using version 4.15.z supports machine pools with version 4.13.z and 4.14.z, but the control plane does not support machine pools using version 4.12.z.
+====
+
+endif::[]
+//END HCP-ONLY conditions
+
+// WHOLE CLUSTER: Condition for upgrading hosted control plane as part of upgrading the whole cluster in sequence
+ifeval::["{context}" == "rosa-hcp-upgrading-whole-cluster"]
+= Upgrading the hosted control plane
+
+When you need to upgrade the whole cluster, upgrade the hosted control plane first.
+endif::[]
+
+
+.Prerequisites
+* You have installed and configured the latest version of the ROSA CLI.
+* No machine pool upgrades are in progress or scheduled to take place at the same time as the hosted control plane upgrade.
+
+//END WHOLE CLUSTER conditions
+
+.Procedure
+
+. Verify the current version of your cluster by running the following command:
++
+[source,terminal]
+----
+$ rosa describe cluster --cluster=<cluster_name_or_id> <1>
+----
+<1> Replace `<cluster_name_or_id>` with the cluster name or the cluster ID.
+
+. List the versions that you can upgrade your control plane to by running the following command:
++
+[source,terminal]
+----
+$ rosa list upgrade --cluster=<cluster_name_or_id>
+----
++
+The command returns a list of available updates, including the recommended version.
++
+.Example output
++
+[source,terminal]
+----
+VERSION  NOTES
+4.14.8   recommended
+4.14.7
+4.14.6
+----
+
+. Upgrade the cluster's hosted control plane by running the following command:
++
+[source,terminal]
+----
+$ rosa upgrade cluster -c <cluster_name_or_id> --control-plane [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
+----
+
+** To schedule an immediate upgrade to the specified version, run the following command:
++
+[source,terminal]
+----
+$ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --version <version_number>
+----
++
+Your hosted control plane is scheduled for an immediate upgrade.
+
+** To schedule an upgrade to the the specified version at a future date, run the following command:
++
+[source,terminal]
+----
+$ rosa upgrade cluster -c <cluster_name_or_id> --control-plane --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version=<version_number>
+----
++
+Your hosted control plane is scheduled for an upgrade at the specified time in Coordinated Universal Time (UTC).
+
+ifeval::["{context}" != "rosa-hcp-upgrading-whole-cluster"]
+.Troubleshooting
+* Sometimes a scheduled upgrade does not initiate. See link:https://access.redhat.com/solutions/6648291[Upgrade maintenance canceled] for more information.
+endif::[]

modules/rosa-hcp-upgrading-cli-machinepool.adoc

@@ -0,0 +1,133 @@
+// Module included in the following assemblies:
+//
+// * upgrading/rosa-hcp-upgrading.adoc
+
+// NOTE: This module is included several times in the same upgrade assembly.
+
+:_mod-docs-content-type: PROCEDURE
+[id="rosa-hcp-upgrading-cli-machinepool_{context}"]
+// POOL-ONLY: Conditions for upgrading machine pools WITHOUT upgrading hosted control planes
+ifeval::["{context}" != "rosa-hcp-upgrading-whole-cluster"]
+= Upgrading machine pools with the ROSA CLI
+
+You can manually upgrade one or more machine pools in a {hcp-title} cluster by using the ROSA CLI. This method schedules the specified machine pools for an upgrade if a more recent version is available, either immediately, or at a specified future time.
+
+[NOTE]
+====
+Your control plane only supports machine pools within two minor Y-stream versions. For example, a {hcp-title} cluster with a control plane using version 4.15.z supports machine pools with version 4.13.z and 4.14.z, but the control plane does not support machine pools using version 4.12.z.
+====
+
+.Prerequisites
+* You have installed and configured the latest version of the ROSA CLI.
+* No upgrades for the hosted control plane are in progress on the cluster, or scheduled to occur at the same time as the machine pool upgrade.
+endif::[]
+//END POOL-ONLY condition
+
+// WHOLE CLUSTER: Conditions for upgrading machine pools as part of upgrading the whole cluster in sequence
+ifeval::["{context}" == "rosa-hcp-upgrading-whole-cluster"]
+= Upgrading machine pools
+
+When your hosted control plane upgrade is complete, you can upgrade one or more machine pools simultaneously.
+endif::[]
+//END WHOLE CLUSTER condition
+
+.Procedure
+. Verify the current version of your cluster by running the following command:
++
+[source,terminal]
+----
+$ rosa describe cluster --cluster=<cluster_name_or_id> <1>
+----
+<1> Replace `<cluster_name_or_id>` with the cluster name or the cluster ID.
++
+ifeval::["{context}" != "rosa-hcp-upgrading-whole-cluster"]
+.Example output
+[source,terminal]
+----
+OpenShift Version:     4.14.0
+----
+endif::[]
+ifeval::["{context}" == "rosa-hcp-upgrading-whole-cluster"]
+.Example output
+[source,terminal]
+----
+OpenShift Version:     4.14.8
+----
+//WHOLE CLUSTER: updating the version here to show after hcp upgrade in whole cluster section
+endif::[]
+
+. List the versions that you can upgrade your machine pools to by running the following command:
++
+[source,terminal]
+----
+$ rosa list upgrade --cluster <cluster-name> --machinepool <machinepool_name>
+----
++
+The command returns a list of available updates, including the recommended version.
++
+.Example output
++
+[source,terminal]
+----
+VERSION  NOTES
+4.14.5   recommended
+4.14.4
+4.14.3
+----
++
+[IMPORTANT]
+====
+Do not upgrade your machine pool to a version higher than your control plane. If you want to move to a higher version, upgrade the control plane to that version first.
+====
+//Is it even possible to do this? Will a higher version display? Can you specify a higher version even if it doesn't display?
+
+. Verify the upgrade behavior of the machine pools you intend to upgrade by running the following command:
++
+[source,terminal]
+----
+$ rosa describe machinepool --cluster=<cluster_name_or_id> <machine_pool_name> 
+----
++
+.Example output
+[source,terminal]
+----
+Replicas: 5
+Node drain grace period:   30 minutes
+
+Management upgrade:
+- Type: Replace
+- Max surge: 20%
+- Max unavailable: 20%
+----
++
+In the example, these settings allow the machine pool to provision one excess node (`max-surge` of 20% of `replicas`) and to have up to one node unavailable (`max-unavailable` of 20% of `replicas`) during an upgrade. This machine pool can therefore upgrade two nodes at a time, by provisioning one new node in excess of the replica count, and by making one node unavailable and replacing it. Node upgrades may be delayed by up to 30 minutes (`node-drain-grace-period` of 30 minutes) if necessary to protect workloads that have a pod disruption budget.
+
+. Upgrade one or more of your machine pools by running the following command:
++
+[source,terminal]
+----
+$ rosa upgrade machinepool -c <cluster_name> <first_machine_pool_id> <second_machine_pool_id> [--schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm>] --version <version_number>
+----
++
+Multiple machine pools can be upgraded simultaneously. You can schedule machine pool upgrades individually or schedule multiple upgrades in a single command.
+
+** To schedule the immediate upgrade of a specific machine pool on your cluster, run the following command:
++
+[source,terminal]
+----
+$ rosa upgrade machinepool -c <cluster_name> <your_machine_pool_id> --version <version_number>
+----
++
+Your machine pool is scheduled for immediate upgrade, which initiates a rolling replacement of all nodes in the specified machine pool.
+
+** To schedule an upgrade of multiple machine pools to start at a future date, run the following command:
++
+[source,terminal]
+----
+$ rosa upgrade machinepool -c <cluster_name> <first_machine_pool_id> <second_machine_pool_id> --schedule-date=<yyyy-mm-dd> --schedule-time=<HH:mm> --version <version_number>
+----
++
+Your machine pools are scheduled to begin an upgrade at the specified time and date in Coordinated Universal Time (UTC). This will initiate a rolling replacement of all nodes in the specified machine pools, beginning at the specified time.
+
+.Troubleshooting
+* Sometimes a scheduled upgrade does not initiate. See link:https://access.redhat.com/solutions/6648291[Upgrade maintenance canceled] for more information.