Skip to content

Commit 8526b6d

Browse files
mburke5678mburke5678
authored
Merge pull request #93880 from mburke5678/mco-on-cluster-layering-reorg
MCO on-cluster layering reorg the using OCL module
2 parents 375162b + 2e98005 commit 8526b6d

File tree

3 files changed

+216
-198
lines changed

3 files changed

+216
-198
lines changed

‎machine_configuration/mco-coreos-layering.adoc

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -177,12 +177,19 @@ It is strongly recommended that you test your images outside of your production
177177
include::modules/coreos-layering-configuring-on.adoc[leveloffset=+1]
178178

179179
.Additional resources
180+
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-proc_mco-coreos-layering[Using on-cluster layering to apply a custom layered image]
180181
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-remove_mco-coreos-layering[Removing an on-cluster custom layered image]
181182
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
182183
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-rebuild_mco-coreos-layering[Rebuilding an on-cluster custom layered image]
183-
* xref:../openshift_images/managing_images/using-image-pull-secrets.adoc#images-update-global-pull-secret_using-image-pull-secrets[Updating the global cluster pull secret]
184+
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-revert_mco-coreos-layering[Reverting an on-cluster custom layered image]
184185
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-modifying_mco-coreos-layering[Modifying a custom layered image]
185186

187+
.Additional resources
188+
include::modules/coreos-layering-configuring-on-proc.adoc[leveloffset=+2]
189+
190+
* xref:../openshift_images/managing_images/using-image-pull-secrets.adoc#images-update-global-pull-secret_using-image-pull-secrets[Updating the global cluster pull secret]
191+
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-revert_mco-coreos-layering[Reverting an on-cluster custom layered image]
192+
186193
include::modules/coreos-layering-configuring-on-modifying.adoc[leveloffset=+2]
187194

188195
.Additional resources

‎modules/coreos-layering-configuring-on-proc.adoc

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/mco-coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on-proc_{context}"]
7+
= Using on-cluster layering to apply a custom layered image
8+
9+
To apply a custom layered image to your cluster by using the on-cluster build process, create a `MachineOSConfig` custom resource (CR) that specifies the following parameters:
10+
11+
* the Containerfile to build
12+
* the machine config pool to associate the build
13+
* where the final image should be pushed and pulled from
14+
* the push and pull secrets to use
15+
16+
.Prerequisites
17+
18+
* You have the pull secret in the `openshift-machine-config-operator` namespace that the Machine Config Operator (MCO) needs in order to pull the base operating system image from your repository. By default, the MCO uses the cluster global pull secret, which it synchronizes into the `openshift-machine-config-operator` namespace. You can add your pull secret to the {product-title} global pull secret or you can use a different pull secret. For information on modifying the global pull secret, see "Updating the global cluster pull secret".
19+
20+
* You have the push secret of the registry that the MCO needs to push the new custom layered image to. The credentials provided by the secret must also grant permission to delete an image from the registry.
21+
+
22+
[NOTE]
23+
====
24+
In a disconnected environment, ensure that the disconnected cluster can access the registry where you want to push the image. Image mirroring applies only to pulling images.
25+
====
26+
27+
* You have the pull secret that your nodes need to pull the new custom layered image from your registry. This should be a different secret than the one used to push the image to the repository.
28+
29+
* You are familiar with how to configure a Containerfile. Instructions on how to create a Containerfile are beyond the scope of this documentation.
30+
31+
* Optional: You have a separate machine config pool for the nodes where you want to apply the custom layered image. One benefit to having a custom machine config pool for the nodes it that you can easily revert to the base image, if needed. For more information, see "Reverting an on-cluster layered node".
32+
33+
.Procedure
34+
35+
. Create a `MachineOSconfig` object:
36+
37+
.. Create a YAML file similar to the following:
38+
+
39+
[source,yaml]
40+
----
41+
apiVersion: machineconfiguration.openshift.io/v1 <1>
42+
kind: MachineOSConfig
43+
metadata:
44+
name: layered-image <2>
45+
spec:
46+
machineConfigPool:
47+
name: layered <3>
48+
containerFile: # <4>
49+
- containerfileArch: NoArch <5>
50+
content: |-
51+
FROM configs AS final
52+
RUN dnf install -y cowsay && \
53+
dnf clean all && \
54+
ostree container commit
55+
imageBuilder: # <6>
56+
imageBuilderType: Job
57+
baseImagePullSecret: # <7>
58+
name: global-pull-secret-copy
59+
renderedImagePushSpec: image-registry.openshift-image-registry.svc:5000/openshift/os-image:latest # <8>
60+
renderedImagePushSecret: # <9>
61+
name: builder-dockercfg-mtcl23
62+
----
63+
<1> Specifies the `machineconfiguration.openshift.io/v1` API that is required for `MachineConfig` CRs.
64+
<2> Specifies a name for the `MachineOSConfig` object. This name is used with other on-cluster layering resources. The examples in this documentation use the name `layered-image`.
65+
<3> Specifies the name of the machine config pool associated with the nodes where you want to deploy the custom layered image. The examples in this documentation use the `layered` machine config pool.
66+
<4> Specifies the Containerfile to configure the custom layered image.
67+
<5> Specifies the architecture this containerfile is to be built for: `ARM64`, `AMD64`, `PPC64LE`, `S390X`, or `NoArch`. The default is `NoArch`, which defines a Containerfile that can be applied to any architecture.
68+
<6> Specifies the name of the image builder to use. This must be `Job`, which is a reference to the `job` object that is managing the image build.
69+
<7> Optional: Specifies the name of the pull secret that the MCO needs to pull the base operating system image from the registry. By default, the global pull secret is used.
70+
<8> Specifies the image registry to push the newly-built custom layered image to. This can be any registry that your cluster has access to in the `host[:port][/namespace]/name` or `svc_name.namespace.svc[:port]/repository/name:<tag>` format. This example uses the internal {product-title} registry. You can specify a mirror registry if you cluster is properly configured to use a mirror registry.
71+
<9> Specifies the name of the push secret that the MCO needs to push the newly-built custom layered image to that registry.
72+
73+
.. Create the `MachineOSConfig` object:
74+
+
75+
[source,terminal]
76+
----
77+
$ oc create -f <filename>.yaml
78+
----
79+
80+
. If necessary, when the `MachineOSBuild` object has been created and is in the `READY` state, modify the node spec for the nodes where you want to use the new custom layered image:
81+
+
82+
.. Check that the `MachineOSBuild` object is ready, by running the following command:
83+
+
84+
[source,terminal]
85+
----
86+
$ oc get machineosbuild
87+
----
88+
+
89+
When the `SUCCEEDED` value is `True`, the build is complete:
90+
+
91+
.Example output showing that the `MachineOSBuild` object is ready
92+
[source,terminal]
93+
----
94+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED AGE
95+
layered-image-ad5a3cad36303c363cf458ab0524e7c0-builder False False True False False 43s
96+
----
97+
98+
.. Edit the nodes where you want to deploy the custom layered image by adding a label for the machine config pool you specified in the `MachineOSConfig` object:
99+
+
100+
[source,terminal]
101+
----
102+
$ oc label node <node_name> 'node-role.kubernetes.io/<mcp_name>='
103+
----
104+
+
105+
--
106+
where:
107+
108+
node-role.kubernetes.io/<mcp_name>=:: Specifies a node selector that identifies the nodes to deploy the custom layered image.
109+
--
110+
+
111+
When you save the changes, the MCO drains, cordons, and reboots the nodes. After the reboot, the node uses the new custom layered image.
112+
113+
.Verification
114+
115+
. Verify that the new pods are ready by running the following command:
116+
+
117+
[source,terminal]
118+
----
119+
$ oc get pods -n openshift-machine-config-operator
120+
----
121+
+
122+
.Example output
123+
[source,terminal]
124+
----
125+
NAME READY STATUS RESTARTS AGE
126+
build-layered-image-ad5a3cad36303c363cf458ab0524e7c0-hxrws 2/2 Running 0 2m40s # <1>
127+
# ...
128+
machine-os-builder-6fb66cfb99-zcpvq 1/1 Running 0 2m42s # <2>
129+
----
130+
<1> This is the build pod where the custom layered image is building, named in the `build-<MachineOSConfig_CR_name>-<hash>` format.
131+
<2> This pod can be used for troubleshooting.
132+
133+
. Verify the current stage of your layered build by running the following command:
134+
+
135+
[source,terminal]
136+
----
137+
$ oc get machineosbuilds
138+
----
139+
+
140+
.Example output
141+
[source,terminal]
142+
----
143+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED AGE
144+
layered-image-ad5a3cad36303c363cf458ab0524e7c0 False True False False False 12m <1>
145+
----
146+
<1> The `MachineOSBuild` is named in the `<MachineOSConfig_CR_name>-<hash>` format.
147+
148+
. Verify that the `MachineOSConfig` object contains a reference to the new custom layered image by running the following command:
149+
+
150+
[source,terminal]
151+
----
152+
$ oc describe machineosconfig <object_name>
153+
----
154+
+
155+
.Example digested image pull spec
156+
[source,terminal]
157+
----
158+
apiVersion: machineconfiguration.openshift.io/v1
159+
kind: MachineOSConfig
160+
metadata:
161+
annotations:
162+
machineconfiguration.openshift.io/current-machine-os-build: layered-9a8f89455246fa0c42ecee6ff1fa1a45
163+
labels:
164+
machineconfiguration.openshift.io/createdByOnClusterBuildsHelper: ""
165+
name: layered-image
166+
# ...
167+
status:
168+
currentImagePullSpec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-image@sha256:3c8fc667adcb432ce0c83581f16086afec08a961dd28fed69bb6bad6db0a0754 <1>
169+
----
170+
<1> Digested image pull spec for the new custom layered image.
171+
172+
. Verify that the appropriate nodes are using the new custom layered image:
173+
174+
.. Start a debug session as root for a control plane node by running the following command:
175+
+
176+
[source,terminal]
177+
----
178+
$ oc debug node/<node_name>
179+
----
180+
181+
.. Set `/host` as the root directory within the debug shell:
182+
+
183+
[source,terminal]
184+
----
185+
sh-4.4# chroot /host
186+
----
187+
188+
.. Run the `rpm-ostree status` command to view that the custom layered image is in use:
189+
+
190+
[source,terminal]
191+
----
192+
sh-5.1# rpm-ostree status
193+
----
194+
+
195+
.Example output
196+
[source,terminal]
197+
----
198+
# ...
199+
Deployments:
200+
* ostree-unverified-registry:image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-images@sha256:3c8fc667adcb432ce0c83581f16086afec08a961dd28fed69bb6bad6db0a0754
201+
Digest: sha256:3c8fc667adcb432ce0c83581f16086afec08a961dd28fed69bb6bad6db0a0754 <1>
202+
Version: 419.94.202502100215-0 (2025-02-12T19:20:44Z)
203+
----
204+
<1> Digested image pull spec for the new custom layered image.

0 commit comments

Comments
 (0)