Skip to content

Commit 202b629

Browse files
kcarmichael08kcarmichael08
authored
Merge pull request #83997 from laubai/osdocs-11411-multiarch-scheduling-post-hcp-merge
OSDOCS-11411: Adding ROSA-compatible multi-arch scheduling content
2 parents f4e1240 + 7f478b0 commit 202b629

10 files changed

+184
-140
lines changed

‎_topic_maps/_topic_map_rosa_hcp.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,8 @@ Topics:
252252
File: nodes-cluster-resource-configure
253253
- Name: Configuring PID limits
254254
File: rosa-configuring-pid-limits
255+
- Name: Managing multi-architecture clusters
256+
File: rosa-multi-arch-cluster-managing
255257
---
256258
Name: Security and compliance
257259
Dir: security

‎modules/multi-architecture-scheduling-examples.adoc

Lines changed: 11 additions & 124 deletions
Original file line numberDiff line numberDiff line change
@@ -4,136 +4,23 @@
44

55
:_mod-docs-content-type: CONCEPT
66
[id="multi-architecture-scheduling-examples_{context}"]
7-
87
= Sample multi-architecture node workload deployments
98

10-
Before you schedule workloads on a cluster with compute nodes of different architectures, consider the following use cases:
9+
Scheduling a workload to an appropriate node based on architecture works in the same way as scheduling based on any other node characteristic.
10+
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
11+
Consider the following options when determining how to schedule your workloads.
12+
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
1113

12-
Using node affinity to schedule workloads on a node:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification.
13-
+
14-
.Example deployment with the `nodeAffinity` set to certain architectures
15-
[source,yaml]
16-
----
17-
apiVersion: apps/v1
18-
kind: Deployment
19-
metadata: # ...
20-
spec:
21-
# ...
22-
template:
23-
# ...
24-
spec:
25-
affinity:
26-
nodeAffinity:
27-
requiredDuringSchedulingIgnoredDuringExecution:
28-
nodeSelectorTerms:
29-
- matchExpressions:
30-
- key: kubernetes.io/arch
31-
operator: In
32-
values: <1>
33-
- amd64
34-
- arm64
35-
----
36-
<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values.
14+
include::snippets/multi-arch-schedule-nodeaffinity.adoc[leveloffset=+0]
3715

38-
Tainting every node for a specific architecture:: You can taint a node to avoid workloads that are not compatible with its architecture to be scheduled on that node. In the case where your cluster is using a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures.
16+
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
3917

40-
* Before you can taint a node, you must scale down the `MachineSet` object or remove available machines. You can scale down the machine set by using one of following commands:
41-
+
42-
[source,terminal]
43-
----
44-
$ oc scale --replicas=0 machineset <machineset> -n openshift-machine-api
45-
----
46-
+
47-
Or:
48-
+
49-
[source,terminal]
50-
----
51-
$ oc edit machineset <machineset> -n openshift-machine-api
52-
----
53-
For more information on scaling machine sets, see "Modifying a compute machine set".
18+
include::snippets/multi-arch-schedule-taint.adoc[leveloffset=+0]
5419

55-
+
56-
--
57-
.Example `MachineSet` with a taint set
58-
[source,yaml]
59-
----
60-
apiVersion: machine.openshift.io/v1beta1
61-
kind: MachineSet
62-
metadata: # ...
63-
spec:
64-
# ...
65-
template:
66-
# ...
67-
spec:
68-
# ...
69-
taints:
70-
- effect: NoSchedule
71-
key: multi-arch.openshift.io/arch
72-
value: arm64
73-
----
74-
You can also set a taint on a specific node by running the following command:
75-
[source,terminal]
76-
----
77-
$ oc adm taint nodes <node-name> multi-arch.openshift.io/arch=arm64:NoSchedule
78-
----
79-
--
20+
include::snippets/multi-arch-schedule-default-toleration-namespace.adoc[leveloffset=+0]
8021

81-
Creating a default toleration:: You can annotate a namespace so all of the workloads get the same default toleration by running the following command:
82-
+
83-
[source,terminal]
84-
----
85-
$ oc annotate namespace my-namespace \
86-
'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multi-arch.openshift.io/arch"}]'
87-
----
22+
include::snippets/multi-arch-schedule-toleration.adoc[leveloffset=+0]
8823

89-
Tolerating architecture taints in workloads:: On a node with a defined taint, workloads will not be scheduled on that node. However, you can allow them to be scheduled by setting a toleration in the pod's specification.
90-
+
91-
.Example deployment with a toleration
92-
[source,yaml]
93-
----
94-
apiVersion: apps/v1
95-
kind: Deployment
96-
metadata: # ...
97-
spec:
98-
# ...
99-
template:
100-
# ...
101-
spec:
102-
tolerations:
103-
- key: "multi-arch.openshift.io/arch"
104-
value: "arm64"
105-
operator: "Equal"
106-
effect: "NoSchedule"
107-
----
108-
+
109-
This example deployment can also be allowed on nodes with the `multi-arch.openshift.io/arch=arm64` taint specified.
24+
include::snippets/multi-arch-schedule-affinity-taint-toleration.adoc[leveloffset=+0]
11025

111-
Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint to the nodes of a specific architecture, the following example toleration is required for scheduling pods.
112-
+
113-
.Example deployment with a node affinity and toleration set.
114-
[source,yaml]
115-
----
116-
apiVersion: apps/v1
117-
kind: Deployment
118-
metadata: # ...
119-
spec:
120-
# ...
121-
template:
122-
# ...
123-
spec:
124-
affinity:
125-
nodeAffinity:
126-
requiredDuringSchedulingIgnoredDuringExecution:
127-
nodeSelectorTerms:
128-
- matchExpressions:
129-
- key: kubernetes.io/arch
130-
operator: In
131-
values:
132-
- amd64
133-
- arm64
134-
tolerations:
135-
- key: "multi-arch.openshift.io/arch"
136-
value: "arm64"
137-
operator: "Equal"
138-
effect: "NoSchedule"
139-
----
26+
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]

‎modules/multi-architecture-scheduling.adoc

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
:_mod-docs-content-type: CONCEPT
2+
[id="multi-architecture-scheduling_{context}"]
3+
= Scheduling workloads on clusters with multi-architecture compute machines
4+
5+
When you deploy workloads on a cluster with compute nodes that use different architectures, you must align pod architecture with the architecture of the underlying node. Your workload may also require additional configuration to particular resources depending on the underlying node architecture.
6+
7+
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
8+
You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pods specifications based on the architectures that the pods can support at creation time.
9+
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]

‎post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc

Lines changed: 11 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,28 +6,23 @@ include::_attributes/common-attributes.adoc[]
66

77
toc::[]
88

9-
== Scheduling workloads on clusters with multi-architecture compute machines
9+
Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you
10+
ifndef::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
11+
configure cluster resource requirements and behavior, or
12+
endif::openshift-dedicated,openshift-rosa,openshift-rosa-hcp[]
13+
schedule workloads in a multi-architecture cluster.
1014

11-
Deploying a workload on a cluster with compute nodes of different architectures requires attention and monitoring of your cluster. There might be further actions you need to take in order to successfully place pods in the nodes of your cluster.
12-
13-
You can use the Multiarch Tuning Operator to enable architecture-aware scheduling of workloads on clusters with multi-architecture compute machines. The Multiarch Tuning Operator implements additional scheduler predicates in the pods specifications based on the architectures that the pods can support at creation time. For more information, see xref:../../post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc#multiarch-tuning-operator[Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator].
14-
15-
For more information on node affinity, scheduling, taints and tolerations, see the following documentation:
16-
17-
ifndef::openshift-dedicated,openshift-rosa[]
18-
* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints].
19-
endif::openshift-dedicated,openshift-rosa[]
20-
21-
* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity]
22-
23-
* xref:../../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler]
15+
include::modules/multi-architecture-scheduling.adoc[leveloffset=+1]
2416

2517
include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2]
2618

2719
.Additional resources
28-
20+
* xref:../../post_installation_configuration/configuring-multi-arch-compute-machines/multiarch-tuning-operator.adoc#multiarch-tuning-operator[Managing workloads on multi-architecture clusters by using the Multiarch Tuning Operator]
21+
* xref:../../nodes/scheduling/nodes-scheduler-taints-tolerations.adoc#nodes-scheduler-taints-tolerations[Controlling pod placement using node taints]
22+
* xref:../../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity]
23+
* xref:../../nodes/scheduling/nodes-scheduler-about.adoc#nodes-scheduler-about[Controlling pod placement using the scheduler]
2924
* xref:../../machine_management/modifying-machineset.adoc#machineset-modifying_modifying-machineset[Modifying a compute machine set]
3025
3126
include::modules/multi-architecture-enabling-64k-pages.adoc[leveloffset=+1]
3227

33-
include::modules/multi-architecture-import-imagestreams.adoc[leveloffset=+1]
28+
include::modules/multi-architecture-import-imagestreams.adoc[leveloffset=+1]

‎rosa_cluster_admin/rosa-multi-arch-cluster-managing.adoc

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
//This assembly duplicates the following file to avoid symbolic links:
3+
// * post_installation_configuration/configuring-multi-arch-compute-machines/multi-architecture-compute-managing.adoc
4+
5+
:context: rosa-multi-arch-compute-managing
6+
[id="rosa-multi-arch-managing"]
7+
= Managing a cluster with multi-architecture compute machines
8+
include::_attributes/common-attributes.adoc[]
9+
10+
toc::[]
11+
12+
Managing a cluster that has nodes with multiple architectures requires you to consider node architecture as you monitor the cluster and manage your workloads. This requires you to take additional considerations into account when you schedule workloads in a multi-architecture cluster.
13+
14+
include::modules/multi-architecture-scheduling.adoc[leveloffset=+1]
15+
16+
include::modules/multi-architecture-scheduling-examples.adoc[leveloffset=+2]
17+
18+
// TODO OSDOCS-11411: Add back when HCP split in Nodes doc is complete
19+
// .Additional resources
20+
// * xref ../nodes/scheduling/nodes-scheduler-node-affinity.adoc#nodes-scheduler-node-affinity[Controlling pod placement on nodes using node affinity]

‎snippets/multi-arch-schedule-affinity-taint-toleration.adoc

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
:_mod-docs-content-type: SNIPPET
2+
3+
Using node affinity with taints and tolerations:: When a scheduler computes the set of nodes to schedule a pod, tolerations can broaden the set while node affinity restricts the set. If you set a taint on nodes that have a specific architecture, you must also add a toleration to workloads that you want to be scheduled there.
4+
+
5+
.Example deployment with node affinity and toleration set
6+
--
7+
[source,yaml]
8+
----
9+
apiVersion: apps/v1
10+
kind: Deployment
11+
metadata: # ...
12+
spec:
13+
# ...
14+
template:
15+
# ...
16+
spec:
17+
affinity:
18+
nodeAffinity:
19+
requiredDuringSchedulingIgnoredDuringExecution:
20+
nodeSelectorTerms:
21+
- matchExpressions:
22+
- key: kubernetes.io/arch
23+
operator: In
24+
values:
25+
- amd64
26+
- arm64
27+
tolerations:
28+
- key: "multiarch.openshift.io/arch"
29+
value: "arm64"
30+
operator: "Equal"
31+
effect: "NoSchedule"
32+
----
33+
--

‎snippets/multi-arch-schedule-default-toleration-namespace.adoc

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
:_mod-docs-content-type: SNIPPET
2+
3+
Creating a default toleration in a namespace:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can annotate a namespace so all of the workloads get the same default toleration by running the following command:
4+
+
5+
.Example default toleration set on a namespace
6+
--
7+
[source,terminal]
8+
----
9+
$ oc annotate namespace my-namespace \
10+
'scheduler.alpha.kubernetes.io/defaultTolerations'='[{"operator": "Exists", "effect": "NoSchedule", "key": "multiarch.openshift.io/arch"}]'
11+
----
12+
--

‎snippets/multi-arch-schedule-nodeaffinity.adoc

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
:_mod-docs-content-type: SNIPPET
2+
3+
Using `nodeAffinity` to schedule nodes with specific architectures:: You can allow a workload to be scheduled on only a set of nodes with architectures supported by its images, you can set the `spec.affinity.nodeAffinity` field in your pod's template specification.
4+
+
5+
.Example deployment with node affinity set
6+
--
7+
[source,yaml]
8+
----
9+
apiVersion: apps/v1
10+
kind: Deployment
11+
metadata: # ...
12+
spec:
13+
# ...
14+
template:
15+
# ...
16+
spec:
17+
affinity:
18+
nodeAffinity:
19+
requiredDuringSchedulingIgnoredDuringExecution:
20+
nodeSelectorTerms:
21+
- matchExpressions:
22+
- key: kubernetes.io/arch
23+
operator: In
24+
values: <1>
25+
- amd64
26+
- arm64
27+
----
28+
<1> Specify the supported architectures. Valid values include `amd64`,`arm64`, or both values.
29+
[source,yaml]
30+
--

‎snippets/multi-arch-schedule-taint.adoc

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
:_mod-docs-content-type: SNIPPET
2+
3+
Tainting each node for a specific architecture:: You can taint a node to avoid the node scheduling workloads that are incompatible with its architecture. When your cluster uses a `MachineSet` object, you can add parameters to the `.spec.template.spec.taints` field to avoid workloads being scheduled on nodes with non-supported architectures.
4+
+
5+
Before you add a taint to a node, you must scale down the `MachineSet` object or remove existing available machines. For more information, see _Modifying a compute machine set_.
6+
+
7+
.Example machine set with taint set
8+
--
9+
[source,yaml]
10+
----
11+
apiVersion: machine.openshift.io/v1beta1
12+
kind: MachineSet
13+
metadata: # ...
14+
spec:
15+
# ...
16+
template:
17+
# ...
18+
spec:
19+
# ...
20+
taints:
21+
- effect: NoSchedule
22+
key: multiarch.openshift.io/arch
23+
value: arm64
24+
----
25+
--
26+
+
27+
You can also set a taint on a specific node by running the following command:
28+
+
29+
[source,terminal]
30+
----
31+
$ oc adm taint nodes <node-name> multiarch.openshift.io/arch=arm64:NoSchedule
32+
----

‎snippets/multi-arch-schedule-toleration.adoc

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
:_mod-docs-content-type: SNIPPET
2+
3+
Tolerating architecture taints in workloads:: When a node or machine set has a taint, only workloads that tolerate that taint can be scheduled. You can configure your workload with a `toleration` so that it is scheduled on nodes with specific architecture taints.
4+
+
5+
.Example deployment with toleration set
6+
--
7+
[source,yaml]
8+
----
9+
apiVersion: apps/v1
10+
kind: Deployment
11+
metadata: # ...
12+
spec:
13+
# ...
14+
template:
15+
# ...
16+
spec:
17+
tolerations:
18+
- key: "multiarch.openshift.io/arch"
19+
value: "arm64"
20+
operator: "Equal"
21+
effect: "NoSchedule"
22+
----
23+
This example deployment can be scheduled on nodes and machine sets that have the `multiarch.openshift.io/arch=arm64` taint specified.
24+
--

0 commit comments

Comments
 (0)