Skip to content

Commit 4055890

Browse files
xenolinuxxenolinux
authored
Merge pull request #89572 from mramendi/RHDEVDOCS-6244
RHDEVDOCS 6244 step actions for cache fetch and upload
2 parents df70aff + 7995c57 commit 4055890

File tree

1 file changed

+174
-3
lines changed

1 file changed

+174
-3
lines changed

‎modules/op-resolver-stepactions-ref.adoc

Lines changed: 174 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,10 @@
44
// // *openshift_pipelines/remote-pipelines-tasks-resolvers.adoc
55
:_mod-docs-content-type: REFERENCE
66
[id="resolver-stepactions-ref_{context}"]
7-
= Step action provided with {pipelines-shortname}
7+
= Step action definitions provided with {pipelines-shortname}
8+
9+
{pipelines-shortname} provides standard `StepAction` definitions that you can use in your tasks. Use the cluster resolver to reference these definitions.
810

9-
{pipelines-shortname} provides a standard `StepAction` definition that you can use in your tasks. Use the cluster resolver to reference this definition.
1011

1112
[discrete]
1213
[id="op-stepaction-git-clone_{context}"]
@@ -23,7 +24,7 @@ kind: Task
2324
metadata:
2425
name: clone-repo-anon
2526
spec:
26-
# ...
27+
# ...
2728
steps:
2829
- name: clone-repo-step
2930
ref:
@@ -75,3 +76,173 @@ spec:
7576
|`URL` |`string` |The URL of the repository that was cloned.
7677
|`COMMITTER_DATE` |`string` |The epoch timestamp of the commit that is at the HEAD of the current branch in the cloned Git repository.
7778
|===
79+
80+
81+
[discrete]
82+
[id="op-stepaction-cache-{context}"]
83+
== cache-upload and cache-fetch
84+
85+
:FeatureName: Using the `cache-upload` and `cache-fetch` step actions
86+
include::snippets/technology-preview.adoc[]
87+
88+
Use the `cache-upload` and `cache-fetch` step actions to preserve the cache directory where a build process keeps its dependencies, storing it in an Amazon Simple Storage Service (S3) bucket, Google Cloud Services (GCS) bucket, or an Open Container Initiative (OCI) repository.
89+
90+
When you use the `cache-upload` step action, the step action calculates a hash based on certain files in your build. You must provide a regular expression to select these files. The `cache-upload` step action stores an image that contains the content of your cache directory, indexed with the hash.
91+
92+
When you use the `cache-fetch` step action, the step action calculates the same hash. Then it checks whether a cached image for this hash is already available. If the image is available, the step action populates your cache directory with the cached content. If the image is not available, the directory remains as it was.
93+
94+
After using the `cache-fetch` step action, you can run the build process. If the cache is successfully fetched, it includes the dependencies that the build process downloaded previously. If the cache was not fetched, the build process downloads dependencies through its normal procedure.
95+
96+
The result of `cache-fetch` indicates whether a cached image was fetched. The subsequent `cache-upload` step action can use the result and skip uploading a new cache image if the cache for the current hash was already available.
97+
98+
The following example task retrieves the source from a repository, fetches the cache (if available), runs a Maven build, and then, if the cache was not fetched, uploads the new cached image of the build directory.
99+
100+
.Example usage of the `cache-fetch` and `cache-upload` step actions in a task
101+
102+
[source,yaml]
103+
----
104+
apiVersion: tekton.dev/v1
105+
kind: Task
106+
metadata:
107+
name: java-demo-task
108+
spec:
109+
workspaces:
110+
- name: source
111+
params:
112+
- name: repo_url
113+
type: string
114+
default: https://github.com/sample-organization/sample-java-project.git
115+
- name: revision
116+
type: string
117+
default: main
118+
- name: registry
119+
type: string
120+
default: image-registry.openshift-image-registry.svc:5000/sample-project/mvn-cache
121+
- name: image
122+
type: string
123+
default: openjdk:latest
124+
- name: buildCommand
125+
type: string
126+
default: "maven -Dmaven.repo.local=${LOCAL_CACHE_REPO} install"
127+
- name: cachePatterns
128+
type: array
129+
default: ["**pom.xml"]
130+
- name: force-cache-upload
131+
type: string
132+
default: "false"
133+
steps:
134+
- name: create-repo
135+
image: $(params.image)
136+
script: |
137+
mkdir -p $(workspaces.source.path)/repo
138+
chmod 777 $(workspaces.source.path)/repo
139+
- name: fetch-repo
140+
ref:
141+
resolver: cluster
142+
params:
143+
- name: name
144+
value: git-clone
145+
- name: namespace
146+
value: openshift-pipelines
147+
- name: kind
148+
value: stepaction
149+
params:
150+
- name: OUTPUT_PATH
151+
value: $(workspaces.source.path)/repo
152+
- name: URL
153+
value: $(params.repo_url)
154+
- name: REVISION
155+
value: $(params.revision)
156+
- name: cache-fetch
157+
ref:
158+
resolver: cluster
159+
params:
160+
- name: name
161+
value: cache-fetch
162+
- name: namespace
163+
value: openshift-pipelines
164+
- name: kind
165+
value: stepaction
166+
params:
167+
- name: patterns
168+
value: $(params.cachePatterns)
169+
- name: source
170+
value: oci://$(params.registry):{{hash}}
171+
- name: cachePath
172+
value: $(workspaces.source.path)/cache
173+
- name: workingdir
174+
value: $(workspaces.source.path)/repo
175+
- name: run-build
176+
image: $(params.image)
177+
workingDir: $(workspaces.source.path)/repo
178+
env:
179+
- name: LOCAL_CACHE_REPO
180+
value: $(workspaces.source.path)/cache/repo
181+
script: |
182+
set -x
183+
$(params.buildCommand)
184+
echo "Cache size is $(du -sh $(workspaces.source.path)/cache)"
185+
- name: cache-upload
186+
ref:
187+
resolver: cluster
188+
params:
189+
- name: name
190+
value: cache-upload
191+
- name: namespace
192+
value: openshift-pipelines
193+
- name: kind
194+
value: stepaction
195+
params:
196+
- name: patterns
197+
value: $(params.cachePatterns)
198+
- name: target
199+
value: oci://$(params.registry):{{hash}}
200+
- name: cachePath
201+
value: $(workspaces.source.path)/cache
202+
- name: workingdir
203+
value: $(workspaces.source.path)/repo
204+
- name: force-cache-upload
205+
value: $(params.force-cache-upload)
206+
----
207+
208+
.Supported parameters for the `cache-fetch` step action
209+
[cols="1,2,1,1",options="header"]
210+
|===
211+
| Parameter | Description | Type | Default value
212+
|`patterns` |Regular expression for selecting files to compute the hash. For example, for a Go project, you can use `go.mod` files to compute the cache, and then the value of this parameter is `pass:[**]/go.sum` (where `pass:[**]` accounts for subdirectories of any depth). | `array` |
213+
|`source` |Source URI for fetching the cache; use `{{hash}}` to specify the cache hash. The supported types are `oci` (example: `oci://quay.io/example-user/go-cache:{{hash}}`) and `s3` (example: `s3://example-bucket/{{hash}}`) | `string` |
214+
|`cachePath` |Path for extracting the cache content. Normally this path is in a workspace. | `string` |
215+
|`workingDir` |Path where the files for calculating the hash are located. | `string` |
216+
|`insecure` |If `"true"`, use insecure mode for fetching the cache. | `string` |`"false"`
217+
|`googleCredentialsPath` |The path where Google credentials are located. Ignored if empty. | `string` |
218+
|`awsConfigFile` |Path to the AWS configuration file. Ignored if empty. | `string` |
219+
|`awsCredentialFile` |Path to the AWS credentials file. Ignored if empty. | `string` |
220+
|`blobQueryParams` |Blob query parameters for configuring S3, GCS, or Azure. Use these optional parameters for additional features such as S3 acceleration, FIPS, or path-style addressing. | `string` |
221+
|===
222+
223+
.Results that the `cache-fetch` step action returns
224+
[cols="1,1,2",options="header"]
225+
|===
226+
| Result | Type | Description
227+
|`fetched` |`string` |`"true"` if the step has fetched the cache or `"false"` if the step has not fetched the cache.
228+
|===
229+
230+
231+
.Supported parameters for the `cache-upload` step action
232+
[cols="1,2,1,1",options="header"]
233+
|===
234+
| Parameter | Description | Type | Default value
235+
|`patterns` |Regular expression for selecting files to compute the hash. For example, for a Go project, you can use `go.mod` files to compute the cache, and then the value of this parameter is `pass:[**]/go.sum` (where `pass:[**]` accounts for subdirectories of any depth). | `array` |
236+
|`target` |Target URI for uploading the cache; use `{{hash}}` to specify the cache hash. The supported types are `oci` (example: `oci://quay.io/example-user/go-cache:{{hash}}`) and `s3` (example: `s3://example-bucket/{{hash}}`) | `string` |
237+
|`cachePath` |Path for cache content, which the step packs into the image. Normally this path is in a workspace. | `string` |
238+
|`workingDir` |Path where the files for calculating the hash are located. | `string` |
239+
|`insecure` |If `"true"`, use insecure mode for uploading the cache. | `string` |`"false"`
240+
|`fetched` |If `"true"`, the cache for this hash was already fetched. | `string` |`"false"`
241+
|`force-cache-upload` |If `"true"`, the step uploads the cache even if it was fetched previously. | `string` |`"false"`
242+
|`googleCredentialsPath` |The path where Google credentials are located. Ignored if empty. | `string` |
243+
|`awsConfigFile` |Path to the AWS configuration file. Ignored if empty. | `string` |
244+
|`awsCredentialFile` |Path to the AWS credentials file. Ignored if empty. | `string` |
245+
|`blobQueryParams` |Blob query parameters for configuring S3, GCS, or Azure. Use these optional parameters for additional features such as S3 acceleration, FIPS, or path-style addressing. | `string` |
246+
|===
247+
248+
The `cache-upload` step action returns no results.

0 commit comments

Comments
 (0)