Skip to content

Commit f6f0183

Browse files
lahinsonlahinson
authored
Merge pull request #80594 from lahinson/osdocs-11001-hcp-migrate-sizing-guide
[OSDOCS-11001]: Migrating HCP sizing content from RHACM to OCP
2 parents 3faaea6 + 026086b commit f6f0183

File tree

6 files changed

+220
-1
lines changed

6 files changed

+220
-1
lines changed

‎hosted_control_planes/hcp-prepare/hcp-sizing-guidance.adoc

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,31 @@ include::_attributes/common-attributes.adoc[]
55
:context: hcp-sizing-guidance
66

77
toc::[]
8+
9+
Many factors, including hosted cluster workload and worker node count, affect how many hosted clusters can fit within a certain number of control-plane nodes. Use this sizing guide to help with hosted cluster capacity planning. This guidance assumes a highly available {hcp} topology. The load-based sizing examples were measured on a bare-metal cluster. Cloud-based instances might have different limiting factors, such as memory size.
10+
11+
You can override the following resource utilization sizing measurements and disable the metric service monitoring.
12+
13+
See the following highly available {hcp} requirements, which were tested with {product-title} version 4.12.9 and later:
14+
15+
* 78 pods
16+
* Three 8 GiB PVs for etcd
17+
* Minimum vCPU: approximately 5.5 cores
18+
* Minimum memory: approximately 19 GiB
19+
20+
[role="_additional-resources"]
21+
.Additional resources
22+
23+
* For more information about disabling the metric service monitoring, see xref:../../hosted_control_planes/hcp-prepare/hcp-override-resource-util.adoc#hcp-override-resource-util[Overriding resource utilization measurements].
24+
* For more information about highly available {hcp} topology, see xref:../../hosted_control_planes/hcp-prepare/hcp-distribute-workloads.adoc#hcp-distribute-workloads[Distributing hosted cluster workloads].
25+
26+
include::modules/hcp-pod-limits.adoc[leveloffset=+1]
27+
28+
[role="_additional-resources"]
29+
.Additional resources
30+
31+
* For more information about supported identity providers, see xref:../../nodes/nodes/nodes-nodes-managing-max-pods.adoc#nodes-nodes-managing-max-pods-proc_nodes-nodes-managing-max-pods[Configuring the maximum number of pods per node] in _Managing the maximum number of pods per node_.
32+
33+
include::modules/hcp-resource-limit.adoc[leveloffset=+1]
34+
include::modules/hcp-load-based-limit.adoc[leveloffset=+1]
35+
include::modules/hcp-sizing-calculation.adoc[leveloffset=+1]

‎hosted_control_planes/index.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ include::_attributes/common-attributes.adoc[]
44
= {hcp-capital} overview
55
:context: hcp-overview
66

7-
You can deploy {product-title} clusters by using two different control plane configurations: standalone or {hcp}. The standalone configuration uses dedicated virtual machines or physical machines to host the control plane. With {hcp} for {product-title}, you create control planes as pods on a hosting cluster without the need for dedicated virtual or physical machines for each control plane.
7+
You can deploy {product-title} clusters by using two different control plane configurations: standalone or {hcp}. The standalone configuration uses dedicated virtual machines or physical machines to host the control plane. With {hcp} for {product-title}, you create control planes as pods on a management cluster without the need for dedicated virtual or physical machines for each control plane.
88

99
toc::[]
1010

‎modules/hcp-load-based-limit.adoc

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
// Module included in the following assemblies:
2+
// * hosted-control-planes/hcp-prepare/hcp-sizing-guidance.adoc
3+
4+
:_mod-docs-content-type: CONCEPT
5+
[id="hcp-load-based-limit_{context}"]
6+
= Load-based limit
7+
8+
The maximum number of {hcp} that the cluster can host is calculated based on the hosted control plane pods CPU and memory utilizations when some workload is put on the hosted control plane Kubernetes API server.
9+
10+
The following method is used to measure the hosted control plane resource utilizations as the workload increases:
11+
12+
* A hosted cluster with 9 workers that use 8 vCPU and 32 GiB each, while using the KubeVirt platform
13+
* The workload test profile that is configured to focus on API control-plane stress, based on the following definition:
14+
15+
+
16+
** Created objects for each namespace, scaling up to 100 namespaces total
17+
+
18+
** Additional API stress with continuous object deletion and creation
19+
+
20+
** Workload queries-per-second (QPS) and Burst settings set high to remove any client-side throttling
21+
22+
As the load increases by 1000 QPS, the hosted control plane resource utilization increases by 9 vCPUs and 2.5 GB memory.
23+
24+
For general sizing purposes, consider the 1000 QPS API rate that is a _medium_ hosted cluster load, and a 2000 QPS API that is a _heavy_ hosted cluster load.
25+
26+
[NOTE]
27+
====
28+
This test provides an estimation factor to increase the compute resource utilization based on the expected API load. Exact utilization rates can vary based on the type and pace of the cluster workload.
29+
====
30+
31+
.Load table
32+
|===
33+
| Hosted control plane resource utilization scaling | vCPUs | Memory (GiB)
34+
35+
| Resource utilization with no load
36+
| 2.9
37+
| 11.1
38+
39+
| Resource utilization with 1000 QPS
40+
| 9.0
41+
| 2.5
42+
|===
43+
44+
As the load increases by 1000 QPS, the hosted control plane resource utilization increases by 9 vCPUs and 2.5 GB memory.
45+
46+
For general sizing purposes, consider a 1000 QPS API rate to be a medium hosted cluster load and a 2000 QPS API to be a heavy hosted cluster load.
47+
48+
The following example shows hosted control plane resource scaling for the workload and API rate definitions:
49+
50+
.API rate table
51+
|===
52+
| QPS (API rate) | vCPU usage | Memory usage (GiB)
53+
54+
| Low load (Less than 50 QPS)
55+
| 2.9
56+
| 11.1
57+
58+
| Medium load (1000 QPS)
59+
| 11.9
60+
| 13.6
61+
62+
| High load (2000 QPS)
63+
| 20.9
64+
| 16.1
65+
|===
66+
67+
The hosted control plane sizing is about control-plane load and workloads that cause heavy API activity, etcd activity, or both. Hosted pod workloads that focus on data-plane loads, such as running a database, might not result in high API rates.

‎modules/hcp-pod-limits.adoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
// Module included in the following assemblies:
2+
// * hosted-control-planes/hcp-prepare/hcp-sizing-guidance.adoc
3+
4+
:_mod-docs-content-type: CONCEPT
5+
[id="hcp-pod-limits_{context}"]
6+
= Pod limits
7+
8+
The `maxPods` setting for each node affects how many hosted clusters can fit in a control-plane node. It is important to note the `maxPods` value on all control-plane nodes. Plan for about 75 pods for each highly available hosted control plane.
9+
10+
For bare-metal nodes, the default `maxPods` setting of 250 is likely to be a limiting factor because roughly three hosted control planes fit for each node given the pod requirements, even if the machine has plenty of resources to spare. Setting the `maxPods` value to 500 by configuring the `KubeletConfig` value allows for greater hosted control plane density, which can help you take advantage of additional compute resources.

‎modules/hcp-resource-limit.adoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
// Module included in the following assemblies:
2+
// * hosted-control-planes/hcp-prepare/hcp-sizing-guidance.adoc
3+
4+
:_mod-docs-content-type: CONCEPT
5+
[id="hcp-resource-limit_{context}"]
6+
= Request-based resource limit
7+
8+
The maximum number of hosted control planes that the cluster can host is calculated based on the hosted control plane CPU and memory requests from the pods.
9+
10+
A highly available hosted control plane consists of 78 pods that request 5 vCPUs and 18 GB memory. These baseline numbers are compared to the cluster worker node resource capacities to estimate the maximum number of hosted control planes.

‎modules/hcp-sizing-calculation.adoc

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
// Module included in the following assemblies:
2+
// * hosted-control-planes/hcp-prepare/hcp-sizing-guidance.adoc
3+
4+
:_mod-docs-content-type: CONCEPT
5+
[id="hcp-sizing-calculation_{context}"]
6+
= Sizing calculation example
7+
8+
This example provides sizing guidance for the following scenario:
9+
10+
* Three bare-metal workers that are labeled as `hypershift.openshift.io/control-plane` nodes
11+
* `maxPods` value set to 500
12+
* The expected API rate is medium or about 1000, according to the load-based limits
13+
14+
.Limit inputs
15+
|===
16+
| Limit description | Server 1 | Server 2
17+
18+
| Number of vCPUs on worker node
19+
| 64
20+
| 128
21+
22+
| Memory on worker node (GiB)
23+
| 128
24+
| 256
25+
26+
| Maximum pods per worker
27+
| 500
28+
| 500
29+
30+
| Number of workers used to host control planes
31+
| 3
32+
| 3
33+
34+
| Maximum QPS target rate (API requests per second)
35+
| 1000
36+
| 1000
37+
|===
38+
39+
.Sizing calculation example
40+
|===
41+
42+
| Calculated values based on worker node size and API rate | Server 1 | Server 2 | Calculation notes
43+
44+
| Maximum {hcp} per worker based on vCPU requests
45+
| 12.8
46+
| 25.6
47+
| Number of worker vCPUs ÷ 5 total vCPU requests per hosted control plane
48+
49+
| Maximum {hcp} per worker based on vCPU usage
50+
| 5.4
51+
| 10.7
52+
| Number of vCPUS ÷ (2.9 measured idle vCPU usage + (QPS target rate ÷ 1000) × 9.0 measured vCPU usage per 1000 QPS increase)
53+
54+
| Maximum {hcp} per worker based on memory requests
55+
| 7.1
56+
| 14.2
57+
| Worker memory GiB ÷ 18 GiB total memory request per hosted control plane
58+
59+
| Maximum {hcp} per worker based on memory usage
60+
| 9.4
61+
| 18.8
62+
| Worker memory GiB ÷ (11.1 measured idle memory usage + (QPS target rate ÷ 1000) × 2.5 measured memory usage per 1000 QPS increase)
63+
64+
| Maximum {hcp} per worker based on per node pod limit
65+
| 6.7
66+
| 6.7
67+
| 500 `maxPods` ÷ 75 pods per hosted control plane
68+
69+
| Minimum of previously mentioned maximums
70+
| 5.4
71+
| 6.7
72+
|
73+
74+
|
75+
| vCPU limiting factor
76+
| `maxPods` limiting factor
77+
|
78+
79+
| Maximum number of {hcp} within a management cluster
80+
| 16
81+
| 20
82+
| Minimum of previously mentioned maximums × 3 control-plane workers
83+
|===
84+
85+
.{hcp-capital} capacity metrics
86+
|===
87+
88+
| Name | Description
89+
90+
| `mce_hs_addon_request_based_hcp_capacity_gauge`
91+
| Estimated maximum number of {hcp} the cluster can host based on a highly available {hcp} resource request.
92+
93+
| `mce_hs_addon_low_qps_based_hcp_capacity_gauge`
94+
| Estimated maximum number of {hcp} the cluster can host if all {hcp} make around 50 QPS to the clusters Kube API server.
95+
96+
| `mce_hs_addon_medium_qps_based_hcp_capacity_gauge`
97+
| Estimated maximum number of {hcp} the cluster can host if all {hcp} make around 1000 QPS to the clusters Kube API server.
98+
99+
| `mce_hs_addon_high_qps_based_hcp_capacity_gauge`
100+
| Estimated maximum number of {hcp} the cluster can host if all {hcp} make around 2000 QPS to the clusters Kube API server.
101+
102+
| `mce_hs_addon_average_qps_based_hcp_capacity_gauge`
103+
| Estimated maximum number of {hcp} the cluster can host based on the existing average QPS of {hcp}. If you do not have an active {hcp}, you can expect low QPS.
104+
|===

0 commit comments

Comments
 (0)