You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
= Using on-cluster layering to apply a custom layered image
7
+
= About {image-mode-os-on-lower}
8
8
9
-
To apply a custom layered image to your cluster by using the on-cluster build process, make a `MachineOSConfig` custom resource (CR) that specifies the following parameters:
9
+
You can use the {image-mode-os-lower} on-cluster build process to apply a custom layered image to your nodes by creating a `MachineOSConfig` custom resource (CR), as described in "Using {image-mode-os-on-caps} to apply a custom layered image".
10
10
11
-
* the Containerfile to build
12
-
* the machine config pool to associate the build
13
-
* where the final image should be pushed and pulled from
14
-
* the push and pull secrets to use
15
-
16
-
When you create the object, the Machine Config Operator (MCO) creates a `MachineOSBuild` object and a builder pod. The build process also creates transient objects, such as config maps, which are cleaned up after the build is complete. The `MachineOSBuild` object and the associated `builder-*` pod use the same naming scheme, `<MachineOSConfig_CR_name>-<hash>`, for example:
11
+
When you create the object, the Machine Config Operator (MCO) creates a `MachineOSBuild` object and a builder pod. The process also creates transient objects, such as config maps, which are cleaned up after the build is complete. The `MachineOSBuild` object and the associated `builder-*` pod use the same naming scheme, `<MachineOSConfig_CR_name>-<hash>`, for example:
17
12
18
13
.Example `MachineOSBuild` object
19
14
[source,terminal]
20
15
----
21
-
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
When the build is complete, the MCO pushes the new custom layered image to your repository and rolled out to the nodes in the associated machine config pool. You can see the digested image pull spec for the new custom layered image in the `MachineOSBuild` object and `machine-os-builder` pod.
27
+
You should not need to interact with these new objects or the `machine-os-builder` pod. However, you can use all of these resources for troubleshooting, if necessary.
28
+
29
+
When the build is complete, the MCO pushes the new custom layered image to your repository and rolls the image out to the nodes in the associated machine config pool. You can see the digested image pull spec for the new custom layered image in the `MachineOSConfig` object. This is now the active image pull spec for this `MachineOSConfig`.
You can test a `MachineOSBuild` object to make sure it builds correctly without rolling out the custom layered image to active nodes by using a custom machine config pool that contains non-production nodes. Alternatively, you can use a custom machine config pool that has no nodes. The `MachineOSBuild` object builds even if there are no nodes for the MCO to deploy the custom layered image onto.
37
50
====
38
51
39
-
You should not need to interact with these new objects or the `machine-os-builder` pod. However, you can use all of these resources for troubleshooting, if necessary.
52
+
You can apply a custom layered image to any machine config pool in your cluster, including the control plane, worker, or custom pools.
40
53
41
-
You need a separate `MachineOSConfig` CR for each machine config pool where you want to use a custom layered image.
54
+
[NOTE]
55
+
====
56
+
For {sno} clusters, you can apply a custom layered image to the control plane node only.
In the case of a build failure, for example due to network issues or an invalid secret, the MCO retries the build three additional times before the job fails. The MCO creates a different build pod for each build attempt. You can use the build pod logs to troubleshoot any build failures. However, the MCO automatically removes these build pods after a short period of time.
61
+
In the case of a build failure, for example due to network issues or an invalid secret, the MCO retries the build three additional times before the job fails. The MCO creates a different build pod for each build attempt. You can use the build pod logs to troubleshoot any build failures. Note that the MCO automatically removes these build pods after a short period of time.
46
62
47
63
.Example failed `MachineOSBuild` object
48
64
[source,terminal]
49
65
----
50
-
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
You can manually rebuild your custom layered image by either modifying your `MachineOSConfig` object or applying an annotation to the `MachineOSConfig` object. For more information, see "Rebuilding an on-cluster custom layered image".
@@ -59,221 +75,17 @@ You can modify an on-custom layered image as needed, to install additional packa
Note the following limitations when working with the on-cluster layering feature:
65
81
66
-
* On-cluster layering is supported only for {product-title} clusters on the AMD64 architecture.
67
-
* On-cluster layering is not supported on multi-architecture compute machines, or {sno} clusters.
82
+
* {image-mode-os-on-caps} is not supported on multi-architecture compute machines.
83
+
* Using multiple `MachineOSConfig` objects on the same machine config pool is not supported. You need a separate `MachineOSConfig` CR for each machine config pool where you want to use a distinct custom layered image.
68
84
* If you scale up a machine set that uses a custom layered image, the nodes reboot two times. The first, when the node is initially created with the base image and a second time when the custom layered image is applied.
69
85
* Node disruption policies are not supported on nodes with a custom layered image. As a result the following configuration changes cause a node reboot:
70
86
** Modifying the configuration files in the `/var` or `/etc` directory
71
87
** Adding or modifying a systemd service
72
88
** Changing SSH keys
73
89
** Removing mirroring rules from `ICSP`, `ITMS`, and `IDMS` objects
74
90
** Changing the trusted CA, by updating the `user-ca-bundle` configmap in the `openshift-config` namespace
75
-
* The images used in creating custom layered images take up space in your push registry. Always be aware of the free space in your registry and prune the images as needed. You can automatically remove an on-cluster custom layered image from the repository by deleting the MachineOSBuild object that created the image. Note that the credentials provided by the registry push secret must also grant permission to delete an image from the registry. For more information, see "Removing an on-cluster custom layered image".
76
-
77
-
.Prerequisites
78
-
79
-
* You have a copy of the pull secret in the `openshift-machine-config-operator` namespace that the MCO needs to pull the base operating system image.
80
-
+
81
-
For example, if you are using the global pull secret, you can run the following command:
// If you are using the global pull secret, the MCO automatically creates a copy when you first create a `MachineOSconfig` object.
93
-
94
-
* You have the push secret of the registry that the MCO needs to push the new custom layered image to.
95
-
96
-
* You have a pull secret that your nodes need to pull the new custom layered image from your registry. This should be a different secret than the one used to push the image to the repository.
97
-
98
-
* You are familiar with how to configure a Containerfile. Instructions on how to create a Containerfile are beyond the scope of this documentation.
99
-
100
-
* Optional: You have a separate machine config pool for the nodes where you want to apply the custom layered image. One benefit to having a custom machine config pool for the nodes it that you can easily revert to the base image, if needed. For more information, see "Reverting an on-cluster layered node".
<1> Specifies the `machineconfiguration.openshift.io/v1` API that is required for `MachineConfig` CRs.
133
-
<2> Specifies a name for the `MachineOSConfig` object. This name is used with other [image-mode-os-on-lower] resources. The examples in this documentation use the name `layered`.
134
-
<3> Specifies the name of the machine config pool associated with the nodes where you want to deploy the custom layered image. The examples in this documentation use the `layered` machine config pool.
135
-
<4> Specifies the Containerfile to configure the custom layered image.
136
-
<5> Specifies the architecture this containerfile is to be built for: `ARM64`, `AMD64`, `PPC64LE`, `S390X`, or `NoArch`. The default is `NoArch`, which defines a Containerfile that can be applied to any architecture.
137
-
<6> Specifies the name of the image builder to use. This must be `Job`, which is a reference to the `job` object that is managing the image build.
138
-
<7> Optional: Specifies the name of the pull secret that the MCO needs to pull the base operating system image from the registry. By default, the global pull secret is used.
139
-
<8> Specifies the image registry to push the newly-built custom layered image to. This can be any registry that your cluster has access to in the `host[:port][/namespace]/name` or `svc_name.namespace.svc[:port]/repository/name:<tag>` format. This example uses the internal {product-title} registry. You can specify a mirror registry if you cluster is properly configured to use a mirror registry.
140
-
<9> Specifies the name of the push secret that the MCO needs to push the newly-built custom layered image to that registry.
141
-
142
-
.. Create the `MachineOSConfig` object:
143
-
+
144
-
[source,terminal]
145
-
----
146
-
$ oc create -f <file_name>.yaml
147
-
----
148
-
149
-
. If necessary, when the `MachineOSBuild` object has been created and is in the `READY` state, modify the node spec for the nodes where you want to use the new custom layered image:
150
-
+
151
-
.. Check that the `MachineOSBuild` object is `READY`. When the `SUCCEEDED` value is `True`, the build is complete.
152
-
+
153
-
[source,terminal]
154
-
----
155
-
$ oc get machineosbuild
156
-
----
157
-
+
158
-
.Example output showing that the `MachineOSBuild` object is ready
159
-
[source,terminal]
160
-
----
161
-
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
.. Edit the nodes where you want to deploy the custom layered image by adding a label for the machine config pool you specified in the `MachineOSConfig` object:
Final Image Pullspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-images@sha256:312e48825e074b01a913deedd6de68abd44894ede50b2d14f99d722f13cda04b <1>
244
-
----
245
-
<1> Digested image pull spec for the new custom layered image.
246
-
247
-
. Verify that the appropriate nodes are using the new custom layered image:
248
-
249
-
.. Start a debug session as root for a control plane node:
250
-
+
251
-
[source,terminal]
252
-
----
253
-
$ oc debug node/<node_name>
254
-
----
255
-
256
-
.. Set `/host` as the root directory within the debug shell:
257
-
+
258
-
[source,terminal]
259
-
----
260
-
sh-4.4# chroot /host
261
-
----
262
-
263
-
.. Run the `rpm-ostree status` command to view that the custom layered image is in use:
<1> Digested image pull spec for the new custom layered image.
91
+
* The images used in creating custom layered images take up space in your push registry. Always be aware of the free space in your registry and prune the images as needed. You can automatically remove an on-cluster custom layered image from the repository by deleting the `MachineOSBuild` object that created the image. Note that the credentials provided by the registry push secret must also grant permission to delete an image from the registry. For more information, see "Removing an on-cluster custom layered image".
0 commit comments