Skip to content

Commit c790bd1

Browse files
mburke5678mburke5678
authored
Merge pull request #87486 from mburke5678/mco-on-cluster-layering-ga
OSDOCS:13346 On Cluster Layering: Phase 3 (GA)
2 parents 32862bc + 111a12e commit c790bd1

6 files changed

+222
-164
lines changed

‎machine_configuration/mco-coreos-layering.adoc

Lines changed: 5 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,6 @@ As soon as you apply the custom layered image to your cluster, you effectively _
3030
There are two methods for deploying a custom layered image onto your nodes:
3131

3232
On-cluster layering:: With xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on_mco-coreos-layering[on-cluster layering], you create a `MachineOSConfig` object where you include the Containerfile and other parameters. The build is performed on your cluster and the resulting custom layered image is automatically pushed to your repository and applied to the machine config pool that you specified in the `MachineOSConfig` object. The entire process is performed completely within your cluster.
33-
+
34-
--
35-
:FeatureName: On-cluster image layering
36-
include::snippets/technology-preview.adoc[]
37-
--
3833

3934
Out-of-cluster layering:: With xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring_mco-coreos-layering[out-of-cluster layering], you create a Containerfile that references an {product-title} image and the RPM that you want to apply, build the layered image in your own environment, and push the image to your repository. Then, in your cluster, create a `MachineConfig` object for the targeted node pool that points to the new image. The Machine Config Operator overrides the base {op-system} image, as specified by the `osImageURL` value in the associated machine config, and boots the new image.
4035

@@ -182,9 +177,11 @@ It is strongly recommended that you test your images outside of your production
182177
include::modules/coreos-layering-configuring-on.adoc[leveloffset=+1]
183178

184179
.Additional resources
185-
* xref:../nodes/clusters/nodes-cluster-enabling-features.adoc#nodes-cluster-enabling[Enabling features using feature gates]
186-
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
187180
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-remove_mco-coreos-layering[Removing an on-cluster custom layered image]
181+
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
182+
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-rebuild_mco-coreos-layering[Rebuilding an on-cluster custom layered image]
183+
* xref:../openshift_images/managing_images/using-image-pull-secrets.adoc#images-update-global-pull-secret_using-image-pull-secrets[Updating the global cluster pull secret]
184+
* xref:../machine_configuration/mco-coreos-layering.adoc#coreos-layering-configuring-on-modifying_mco-coreos-layering[Modifying a custom layered image]
188185

189186
include::modules/coreos-layering-configuring-on-modifying.adoc[leveloffset=+2]
190187

@@ -200,8 +197,7 @@ include::modules/coreos-layering-configuring-on-extensions.adoc[leveloffset=+2]
200197
* xref:../updating/updating_a_cluster/update-using-custom-machine-config-pools.adoc#update-using-custom-machine-config-pools-pause_update-using-custom-machine-config-pools[Pausing the machine config pools]
201198
////
202199

203-
// Not in 4.18; maybe 4.19
204-
// include::modules/coreos-layering-configuring-on-rebuild.adoc[leveloffset=+2]
200+
include::modules/coreos-layering-configuring-on-rebuild.adoc[leveloffset=+2]
205201

206202
include::modules/coreos-layering-configuring-on-revert.adoc[leveloffset=+2]
207203

‎modules/coreos-layering-configuring-on-modifying.adoc

Lines changed: 24 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -24,42 +24,37 @@ include::snippets//coreos-layering-configuring-on-pause.adoc[]
2424
+
2525
[source,yaml]
2626
----
27-
apiVersion: machineconfiguration.openshift.io/v1alpha1
27+
apiVersion: machineconfiguration.openshift.io/v1
2828
kind: MachineOSConfig
2929
metadata:
30-
name: layered
30+
name: layered-image
3131
spec:
3232
machineConfigPool:
3333
name: worker
34-
buildInputs:
35-
containerFile:
36-
- containerfileArch: noarch
37-
content: |- <1>
38-
FROM configs AS final
39-
40-
RUN rpm-ostree install rng-tools && \
41-
systemctl enable rngd && \
42-
rpm-ostree cleanup -m && \
43-
ostree container commit
44-
45-
RUN rpm-ostree install tree && \
46-
ostree container commit
47-
imageBuilder:
48-
imageBuilderType: PodImageBuilder
49-
baseImagePullSecret:
50-
name: global-pull-secret-copy <2>
51-
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-images:latest <3>
52-
renderedImagePushSecret: <4>
53-
name: new-secret-name
54-
buildOutputs:
55-
currentImagePullSecret:
56-
name: new-secret-name <5>
34+
containerFile:
35+
- containerfileArch: noarch
36+
content: |- <1>
37+
FROM configs AS final
38+
39+
RUN rpm-ostree install rng-tools && \
40+
systemctl enable rngd && \
41+
rpm-ostree cleanup -m && \
42+
ostree container commit
43+
44+
RUN rpm-ostree install tree && \
45+
ostree container commit
46+
imageBuilder:
47+
imageBuilderType: PodImageBuilder
48+
baseImagePullSecret:
49+
name: global-pull-secret-copy <2>
50+
renderedImagePushspec: image-registry.openshift-image-registry.svc:5000/openshift-machine-config-operator/os-images:latest <3>
51+
renderedImagePushSecret: <4>
52+
name: new-secret-name
5753
----
5854
<1> Optional: Modify the Containerfile, for example to add or remove packages.
5955
<2> Optional: Update the secret needed to pull the base operating system image from the registry.
6056
<3> Optional: Modify the image registry to push the newly built custom layered image to.
6157
<4> Optional: Update the secret needed to push the newly built custom layered image to the registry.
62-
<5> Optional: Update the secret needed to pull the newly built custom layered image from the registry.
6358
+
6459
When you save the changes, the MCO drains, cordons, and reboots the nodes. After the reboot, the node uses the cluster base {op-system-first} image. If your changes modify a secret only, no new build is triggered and no reboot is performed.
6560

@@ -75,9 +70,9 @@ $ oc get machineosbuild
7570
.Example output
7671
[source,terminal]
7772
----
78-
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED
79-
layered-a5457b883f5239cdcb71b57e1a30b6ef False False True False False
80-
layered-f91f0f5593dd337d89bf4d38c877590b False True False False False <1>
73+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED AGE
74+
layered-image-a5457b883f5239cdcb71b57e1a30b6ef False False True False False 4d17h
75+
layered-image-f91f0f5593dd337d89bf4d38c877590b False True False False False 2m41s <1>
8176
----
8277
<1> The value `True` in the `BUILDING` column indicates that the `MachineOSBuild` object is building. When the `SUCCEEDED` column reports `True`, the build is complete.
8378

‎modules/coreos-layering-configuring-on-rebuild.adoc

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/coreos-layering.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="coreos-layering-configuring-on-rebuild_{context}"]
7+
= Rebuilding an on-cluster custom layered image
8+
9+
In situations where you want to rebuild a custom layered image, you can either modify your `MachineOSConfig` object or add an annotation to the `MachineOSConfig` object. Both of these actions trigger an automatic rebuild of the object. For example, you could perform a rebuild if the you change the Containerfile or need to update the `osimageurl` location in a machine config.
10+
11+
After you add the annotation, the Machine Config Operator (MCO) deletes the current `MachineOSBuild` object and creates a new one in its place. When the build process is complete, the MCO automatically removes the annotation.
12+
13+
.Prerequisites
14+
15+
* You have opted-in to on-cluster layering by creating a `MachineOSConfig` object.
16+
17+
.Procedure
18+
19+
* Edit the `MachineOSConfig` object to add the `machineconfiguration.openshift.io/rebuild` annotation by using the following command:
20+
+
21+
[source,terminal]
22+
----
23+
$ oc edit MachineOSConfig <object_name>
24+
----
25+
+
26+
.Example `MachineOSConfig` object
27+
+
28+
[source,yaml]
29+
----
30+
apiVersion: machineconfiguration.openshift.io/v1
31+
kind: MachineOSConfig
32+
metadata:
33+
annotations:
34+
machineconfiguration.openshift.io/current-machine-os-build: layering-c26d4a003432df70ee66c83981144cfa
35+
machineconfiguration.openshift.io/rebuild: "" <1>
36+
# ...
37+
name: layered-image
38+
# ...
39+
----
40+
<1> Add this annotation to trigger a rebuild of the custom layered image.
41+
42+
.Verification
43+
44+
* Check that the `MachineOSBuild` object is building by using the following command:
45+
+
46+
[source,terminal]
47+
----
48+
$ oc get machineosbuild
49+
----
50+
+
51+
.Example output
52+
[source,terminal]
53+
----
54+
NAME PREPARED BUILDING SUCCEEDED INTERRUPTED FAILED AGE
55+
layered-image-d6b929a29c6dbfa8e4007c8069a2fd08 False True False False False 2m41s <1>
56+
----
57+
<1> The value `True` in the `BUILDING` column indicates that the `MachineOSBuild` object is building.
58+
59+
* Edit the `MachineOSConfig` object to verify that the MCO removed the `machineconfiguration.openshift.io/rebuild` annotation by using the following command:
60+
+
61+
[source,terminal]
62+
----
63+
$ oc edit MachineOSConfig <object_name>
64+
----
65+
+
66+
.Example `MachineOSConfig` object
67+
+
68+
[source,yaml]
69+
----
70+
apiVersion: machineconfiguration.openshift.io/v1
71+
kind: MachineOSConfig
72+
metadata:
73+
annotations:
74+
machineconfiguration.openshift.io/current-machine-os-build: layering-c26d4a003432df70ee66c83981144cfa
75+
# ...
76+
name: layered-image
77+
# ...
78+
----

‎modules/coreos-layering-configuring-on-revert.adoc

Lines changed: 26 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -6,18 +6,19 @@
66
[id="coreos-layering-configuring-on-revert_{context}"]
77
= Reverting an on-cluster custom layered image
88

9-
You can revert an on-cluster custom layered image from nodes by removing the label for the machine config pool (MCP) that you specified in the `MachineOSConfig` object. After you remove the label, the Machine Config Operator (MCO) reboots the nodes in that MCP with the cluster base {op-system-first} image, along with any previously-made machine config changes, overriding the custom layered image.
9+
If you applied an on-cluster layered image to a node in a custom machine config pool (MCP), you can remove the custom layered image from the node and revert to the base image.
10+
11+
To revert the node, remove the node from the custom MCP by removing the custom machine config pool label from the node. After you remove the label, the Machine Config Operator (MCO) reboots the node with the cluster base {op-system-first} image, overriding the custom layered image.
1012

1113
[IMPORTANT]
1214
====
13-
If the node where the custom layered image is deployed uses a custom machine config pool, before you remove the label, make sure the node is associated with a second MCP.
15+
Before you remove the label, make sure the node is associated with another MCP.
1416
====
1517

16-
You can reapply the custom layered image to the node by using the `oc label node/<node_name> 'node-role.kubernetes.io/<mcp_name>='` label.
17-
1818
.Prerequisites
1919

20-
* You have opted in to on-cluster layering by creating a `MachineOSConfig` object.
20+
* You have opted-in to on-cluster layering by creating a `MachineOSConfig` object.
21+
* You have applied a `MachineOSConfig` object to a node in a custom machine config pool.
2122
2223
.Procedure
2324

@@ -32,9 +33,9 @@ When you save the changes, the MCO drains, cordons, and reboots the nodes. After
3233
3334
.Verification
3435

35-
You can verify that the custom layered image is removed by performing the following checks:
36+
* Verify that the custom layered image is removed by performing any of the following checks:
3637
37-
. Check that the worker machine config pool is updating with the previous machine config:
38+
** Check that the worker machine config pool is updating with the previous machine config:
3839
+
3940
[source,terminal]
4041
----
@@ -45,12 +46,14 @@ $ oc get mcp
4546
[source,terminal]
4647
----
4748
NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE
48-
master rendered-master-8332482204e0b76002f15ecad15b6c2d True False False 3 3 3 0 5h26m
49-
worker rendered-worker-bde4e4206442c0a48b1a1fb35ba56e85 False True False 3 2 2 0 5h26m <1>
49+
layered rendered-layered-e8c8bc1de69777325003e80bc0c04b82 True False False 0 0 0 0 4h20m <1>
50+
master rendered-master-50d7bc27ee8b9ca2250383f0647ade7f True False False 3 3 3 0 5h39m
51+
worker rendered-worker-e8c8bc1de69777325003e80bc0c04b82 True False False 3 3 3 0 5h39m <2>
5052
----
51-
<1> The value `FALSE` in the `UPDATED` column indicates that the `MachineOSBuild` object is building. When the `UPDATED` column reports `FALSE`, the base image has rolled out to the nodes.
53+
<1> The custom machine config pool no longer has any nodes.
54+
<2> When the `UPDATING` field is `True`, the machine config pool is updating with the previous machine config. When the field becomes `False`, the worker machine config pool has rolled out to the previous machine config.
5255

53-
. Check the nodes to see that scheduling on the nodes is disabled. This indicates that the change is being applied:
56+
** Check the nodes to see that scheduling on the nodes is disabled. This indicates that the change is being applied:
5457
+
5558
[source,terminal]
5659
----
@@ -69,36 +72,35 @@ ip-10-0-211-49.us-west-1.compute.internal Ready control-
6972
ip-10-0-218-151.us-west-1.compute.internal Ready worker 31m v1.32.3
7073
----
7174

72-
. When the node is back in the `Ready` state, check that the node is using the base image:
73-
74-
.. Open an `oc debug` session to the node. For example:
75+
** When the node is back in the `Ready` state, check that the node is using the base image:
76+
+
77+
. Open an `oc debug` session to the node. For example:
7578
+
7679
[source,terminal]
7780
----
78-
$ oc debug node/<node_name>
81+
$ oc debug node/ip-10-0-155-125.us-west-1.compute.internal
7982
----
80-
81-
.. Set `/host` as the root directory within the debug shell:
83+
+
84+
. Set `/host` as the root directory within the debug shell:
8285
+
8386
[source,terminal]
8487
----
85-
sh-5.1# chroot /host
88+
sh-4.4# chroot /host
8689
----
8790

88-
.. Run an `rpm-ostree status` command to view that the base image is in use:
91+
. Run the `rpm-ostree status` command to view that the base image is in use:
8992
+
9093
[source,terminal]
9194
----
92-
sh-5.1# rpm-ostree status
95+
sh-4.4# rpm-ostree status
9396
----
9497
+
9598
.Example output
9699
+
97-
[source,terminal]
98100
----
99101
State: idle
100102
Deployments:
101-
* ostree-unverified-image:containers-storage:quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:76721c875a2b79688be46b1dca654c2c6619a6be28b29a2822cd86c3f9d8e3c1
102-
Digest: sha256:76721c875a2b79688be46b1dca654c2c6619a6be28b29a2822cd86c3f9d8e3c1
103-
Version: 418.94.202501300706-0 (2025-01-30T07:10:58Z)
103+
* ostree-unverified-registry:registry.build05.ci.openshift.org/ci-ln-qd0hmqk/stable@sha256:a8bd32573f787f6d1c23e1d669abbefd1e31339826d06e750c0ca632ad6c414f
104+
Digest: sha256:a8bd32573f787f6d1c23e1d669abbefd1e31339826d06e750c0ca632ad6c414f
105+
Version: 419.96.202501202201-0 (2025-01-20T22:06:13Z)
104106
----

0 commit comments

Comments
 (0)