Skip to content

Commit 76dc159

Browse files
author
Michael Burke
committed
OSDOCS:14467 [GA] Pin and preload images
1 parent 9a721ce commit 76dc159

File tree

3 files changed

+181
-0
lines changed

3 files changed

+181
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2352,6 +2352,8 @@ Topics:
23522352
File: machine-config-node-disruption
23532353
- Name: Configuring MCO-related custom resources
23542354
File: machine-configs-custom
2355+
- Name: About pinned images
2356+
File: machine-config-pin-preload-images-about
23552357
- Name: Updated boot images
23562358
File: mco-update-boot-images
23572359
- Name: Managing unused rendered machine configs

‎machine_configuration/machine-config-pin-preload-images-about.adoc

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="machine-config-pin-preload-images-about"]
3+
= About pinned images
4+
include::_attributes/common-attributes.adoc[]
5+
:context: machine-config-operator
6+
7+
toc::[]
8+
9+
A slow, unreliable connection to an image registry can interfere with operations that require pulling images, such as updating a cluster or deploying an application. This can include clusters that have low bandwidth, clusters with unreliable internet connectivity, or clusters in a disconnected environment. For example, a cluster update might require pulling more than one hundred images. Failure to pull those images could cause retries that can interfere with the update process and might cause the update to fail.
10+
11+
One way to prevent this is to pull the required images in advance, before they are actually needed, and _pinning_ those images to specific machine config pools. This ensures that the images are available to your nodes when needed. Pinned images can provide a more consistent update, which is important when scheduling updates into maintenance windows.
12+
13+
Pinned images also ensures that the images are available when deploying applications, so that you can deploy in a more reliable manner.
14+
15+
You can pin images to specific nodes by using a `PinnedImageSet` custom resource (CR), as described in _Pinning images_. Pinned images are stored on the nodes in the `/etc/crio/crio.conf.d/50-pinned-images` file on those nodes. The contents of the file appear similar to the following example:
16+
17+
[source,terminal]
18+
----
19+
[crio]
20+
[crio.image]
21+
pinned_images = ["quay.io/openshift-release-dev/ocp-release@sha256:4198606580b69c8335ad7ae531c3a74e51aee25db5faaf368234e8c8dae5cbea", "quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e", "quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b"]
22+
----
23+
24+
Another benefit to pinned images is that image garbage collection does not remove the pinned images.
25+
26+
Before pulling the images, the Machine Config Operator (MCO) verifies that there is enough storage space available on each affected node. If the node has sufficient space, the MCO creates the pinned image file, pulls the images, and reloads CRI-O. If there is not sufficient space, the MCO does not pull the images and presents an error message.
27+
28+
include::modules/machine-config-pin-preload-images.adoc[leveloffset=+1]
29+
30+
////
31+
Add link to machine config nodes when available
32+
[role="_additional-resources"]
33+
.Additional resources
34+
* xref:
35+
////

‎modules/machine-config-pin-preload-images.adoc

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/machine-config-pin-preload-images-about.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="machine-config-pin-preload-images_{context}"]
7+
= Pinning images
8+
9+
You can pin images to your nodes by using a `PinnedImageSet` custom resource (CR). The pinned image set defines the list of images to pre-load and the machine config pool to which the images should be pinned.
10+
11+
The images are stored in the the `/etc/crio/crio.conf.d/50-pinned-images` file on the nodes.
12+
13+
.Procedure
14+
15+
. Create a YAML file that defines the `PinnedImageSet` object, similar to the following example:
16+
+
17+
[source,yaml]
18+
----
19+
apiVersion: machineconfiguration.openshift.io/v1
20+
kind: PinnedImageSet
21+
metadata:
22+
labels: <1>
23+
machineconfiguration.openshift.io/role: worker
24+
name: worker-pinned-images
25+
spec:
26+
pinnedImages: <2>
27+
- name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
28+
- name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
29+
----
30+
+
31+
where:
32+
+
33+
--
34+
`labels`:: Specifies an optional node selector to specify the machine config pool to pin the images to. If not specified, the images are pinned to all nodes in the cluster.
35+
`pinnedImages`:: Specifies a list of one or more images to pre-load.
36+
--
37+
38+
. Create the `PinnedImageSet` object by running the following command:
39+
+
40+
[source,terminal]
41+
----
42+
$ oc create -f <file_name>.yaml
43+
----
44+
45+
.Verification
46+
47+
* Check that the pinned image set is reported in the machine config node object for the affected machine config pool by running the following command:
48+
+
49+
[source,terminal]
50+
----
51+
$ oc describe machineconfignode <machine_config_node_name>
52+
----
53+
+
54+
.Example command
55+
[source,terminal]
56+
----
57+
$ oc describe machineconfignode ci-ln-25hlkvt-72292-jrs48-worker-a-2bdj
58+
----
59+
+
60+
.Example output for a successful image pull and pin
61+
+
62+
[source,terminal]
63+
----
64+
apiVersion: machineconfiguration.openshift.io/v1
65+
kind: MachineConfigNode
66+
metadata:
67+
creationTimestamp: "2025-04-28T18:40:29Z"
68+
generation: 3
69+
name: <machine_config_node_name>
70+
# ...
71+
status
72+
pinnedImageSets:
73+
- currentGeneration: 1
74+
desiredGeneration: 1
75+
name: worker-pinned-images <1>
76+
----
77+
<1> The `PinnedImageset` object is associated with the machine config node.
78+
+
79+
Any failures or error messages would appear in the `MachineConfigNode` object status fields, as shown in the following example:
80+
+
81+
.Example output for a failed image pull and pin
82+
+
83+
[source,terminal]
84+
----
85+
apiVersion: machineconfiguration.openshift.io/v1
86+
kind: MachineConfigNode
87+
metadata:
88+
creationTimestamp: "2025-04-28T18:40:29Z"
89+
generation: 3
90+
name: <machine_config_node_name>
91+
# ...
92+
- lastTransitionTime: "2025-04-29T19:37:23Z"
93+
message: One or more PinnedImageSet is experiencing an error. See PinnedImageSet
94+
list for more details
95+
reason: PrefetchFailed
96+
status: "True"
97+
type: PinnedImageSetsDegraded
98+
configVersion:
99+
current: rendered-worker-cef1b52c532e19a20add12e369261fba
100+
desired: rendered-worker-cef1b52c532e19a20add12e369261fba
101+
observedGeneration: 3
102+
pinnedImageSets:
103+
- desiredGeneration: 1
104+
lastFailedGeneration: 1
105+
lastFailedGenerationError: 'failed to execute podman manifest inspect for "quay.io/rh-ee/machine-config-operator@sha256:65d3a308767b1773b6e3499dde6ef085753d7e20e685f78841079":
106+
exit status 125'
107+
name: worker-pinned-images
108+
----
109+
110+
* Check that the pinned image file is created and contains the correct images.
111+
112+
.. Start a debug session for a node by running the following command:
113+
+
114+
[source,terminal]
115+
----
116+
$ oc debug node/<node_name>
117+
----
118+
119+
.. Set `/host` as the root directory within the debug shell by running the following command:
120+
+
121+
[source,terminal]
122+
----
123+
sh-5.1# chroot /host
124+
----
125+
126+
.. Verify the contents of the pinned image file by running the following command:
127+
+
128+
[source,terminal]
129+
----
130+
$ cat /etc/crio/crio.conf.d/50-pinned-images
131+
----
132+
+
133+
.Example output
134+
+
135+
[source,terminal]
136+
----
137+
[crio]
138+
[crio.image]
139+
pinned_images = ["quay.io/openshift-release-dev/ocp-release@sha256:4198606580b69c8335ad7ae531c3a74e51aee25db5faaf368234e8c8dae5cbea", "quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e", "quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b"] <1>
140+
----
141+
+
142+
where:
143+
+
144+
`pinnedImages`:: Specifies the images that have been pulled and pinned for the affected machine config pool.

0 commit comments

Comments
 (0)