Skip to content

Commit cba21c2

Browse files
author
Michael Burke
committed
OSDOCS:14467 [GA] Pin and preload images
1 parent 9a721ce commit cba21c2

File tree

3 files changed

+174
-0
lines changed

3 files changed

+174
-0
lines changed

‎_topic_maps/_topic_map.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2352,6 +2352,8 @@ Topics:
23522352
File: machine-config-node-disruption
23532353
- Name: Configuring MCO-related custom resources
23542354
File: machine-configs-custom
2355+
- Name: About preloading and pinning images
2356+
File: machine-config-pin-preload-images-about
23552357
- Name: Updated boot images
23562358
File: mco-update-boot-images
23572359
- Name: Managing unused rendered machine configs

‎machine_configuration/machine-config-pin-preload-images-about.adoc

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
[id="machine-config-pin-preload-images-about"]
3+
= About pinning and preloading images
4+
include::_attributes/common-attributes.adoc[]
5+
:context: machine-config-operator
6+
7+
toc::[]
8+
9+
Slow and unreliable connections to an image registry can interfere with operations that require pulling images, such as updating a cluster or deploying an application. This can include clusters that have low bandwidth, clusters with unreliable internet connectivity, or clusters in a disconnected environment.
10+
11+
For example, a cluster update might require pulling more than one hundred images. Failure to pull those images could cause retries that can interfere with the update process and might cause the update to fail. One way to improve that is to _preload_ the required images by pulling them in advance, before they are actually needed. Then, by _pinning_ them to a machine config pool, you ensure that the images are available when needed. Pinning and preloading images can provide a more consistent update, which is important when scheduling updates into maintenance windows.
12+
13+
Pinning and preloading images also ensures that the images are available when deploying applications, which allows you to deploy in a predictable time. Another benefit to pinning and preloading images is that image garbage collection does not remove the pinned images.
14+
15+
You can pin and preload images by using a a `PinnedImageSet` custom resource (CR) as described in the following section. Pinned images are stored on the nodes in the `/etc/crio/crio.conf.d/50-pinned-images` file. The contents of the file appear similar to the following example:
16+
17+
[source,terminal]
18+
----
19+
[crio]
20+
[crio.image]
21+
pinned_images = ["quay.io/openshift-release-dev/ocp-release@sha256:4198606580b69c8335ad7ae531c3a74e51aee25db5faaf368234e8c8dae5cbea", "quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e", "quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b"]
22+
----
23+
24+
Before pulling the images, the Machine Config Operator (MCO) verifies that there is enough space available on each affected node to store the images. If the node has sufficient space, the MCO creates the pinned image file, pulls the images, and reloads CRI-O. If there is not sufficient space, the MCO does not pull the images and presents an error message.
25+
26+
include::modules/machine-config-pin-preload-images.adoc[leveloffset=+1]
27+
28+
////
29+
Add link to machine config nodes when available
30+
[role="_additional-resources"]
31+
.Additional resources
32+
* xref:
33+
////

‎modules/machine-config-pin-preload-images.adoc

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
// Module included in the following assemblies:
2+
//
3+
// * machine_configuration/machine-config-pin-preload-images-about.adoc
4+
5+
:_mod-docs-content-type: PROCEDURE
6+
[id="machine-config-pin-preload-images_{context}"]
7+
= Pinning and preloading images
8+
9+
You can pin and preload images by using a `PinnedImageSet` custom resource (CR). The pinned image set defines the list of images to preload and the machine config pool to which the images should be pinned.
10+
11+
The images are stored in the the `/etc/crio/crio.conf.d/50-pinned-images` file on the nodes.
12+
13+
.Procedure
14+
15+
. Create a YAML file that defines the `PinnedImageSet` similar to the following example:
16+
+
17+
[source,yaml]
18+
----
19+
apiVersion: machineconfiguration.openshift.io/v1
20+
kind: PinnedImageSet
21+
metadata:
22+
labels: <1>
23+
machineconfiguration.openshift.io/role: worker
24+
name: worker-pinned-images
25+
spec:
26+
pinnedImages: <2>
27+
- name: quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e
28+
- name: quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b
29+
----
30+
+
31+
where:
32+
+
33+
--
34+
`labels`:: Specifies an optional node selector to specify the machine config pool to pin the images to. If not specified, the images are pinned to all nodes in the cluster.
35+
`pinnedImages`:: Specifies a list of one or more images to preload.
36+
--
37+
38+
. Create the `PinnedImageSet` object by running the following command:
39+
+
40+
[source,terminal]
41+
----
42+
$ oc create -f <file_name>.yaml
43+
----
44+
45+
.Verification
46+
47+
* Check that the pinned image set is reported in the machine config node object for the affected machine config pool by running the following command:
48+
+
49+
[source,terminal]
50+
----
51+
$ oc describe machineconfignode <machine_confignode_name>
52+
----
53+
+
54+
.Example command
55+
[source,terminal]
56+
----
57+
$ oc describe machineconfignode ci-ln-25hlkvt-72292-jrs48-worker-a-2bdj
58+
----
59+
+
60+
.Example output for a successful image pull and pin
61+
+
62+
[source,terminal]
63+
----
64+
apiVersion: machineconfiguration.openshift.io/v1
65+
kind: MachineConfigNode
66+
metadata:
67+
creationTimestamp: "2025-04-28T18:40:29Z"
68+
generation: 3
69+
name: <machine_config_node_name>
70+
# ...
71+
status
72+
pinnedImageSets:
73+
- currentGeneration: 1
74+
desiredGeneration: 1
75+
name: worker-pinned-images
76+
----
77+
+
78+
Any failures or error messages would appear in the `MachineConfigNode` object status fields, as shown in the following example:
79+
+
80+
.Example output for a failed image pull and pin
81+
+
82+
[source,terminal]
83+
----
84+
apiVersion: machineconfiguration.openshift.io/v1
85+
kind: MachineConfigNode
86+
metadata:
87+
creationTimestamp: "2025-04-28T18:40:29Z"
88+
generation: 3
89+
name: <machine_config_node_name>
90+
# ...
91+
- lastTransitionTime: "2025-04-29T19:37:23Z"
92+
message: One or more PinnedImageSet is experiencing an error. See PinnedImageSet
93+
list for more details
94+
reason: PrefetchFailed
95+
status: "True"
96+
type: PinnedImageSetsDegraded
97+
configVersion:
98+
current: rendered-worker-cef1b52c532e19a20add12e369261fba
99+
desired: rendered-worker-cef1b52c532e19a20add12e369261fba
100+
observedGeneration: 3
101+
pinnedImageSets:
102+
- desiredGeneration: 1
103+
lastFailedGeneration: 1
104+
lastFailedGenerationError: 'failed to execute podman manifest inspect for "quay.io/rh-ee/machine-config-operator@sha256:65d3a308767b1773b6e3499dde6ef085753d7e20e685f78841079":
105+
exit status 125'
106+
name: worker-pinned-images
107+
----
108+
109+
* Check that the pinned image file is created and contains the correct images. The contents of the file appear similar to the following example:
110+
111+
.. Start a debug session for a node by running the following command:
112+
+
113+
[source,terminal]
114+
----
115+
$ oc debug node/<node_name>
116+
----
117+
118+
.. Set `/host` as the root directory within the debug shell by running the following command:
119+
+
120+
[source,terminal]
121+
----
122+
sh-5.1# chroot /host
123+
----
124+
125+
.. Verify the contents of the pinned image file by running the following command:
126+
+
127+
[source,terminal]
128+
----
129+
$ cat /etc/crio/crio.conf.d/50-pinned-images
130+
----
131+
+
132+
.Example output
133+
+
134+
[source,terminal]
135+
----
136+
[crio]
137+
[crio.image]
138+
pinned_images = ["quay.io/openshift-release-dev/ocp-release@sha256:4198606580b69c8335ad7ae531c3a74e51aee25db5faaf368234e8c8dae5cbea", "quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e", "quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b"]
139+
----

0 commit comments

Comments
 (0)