Skip to content

Commit b4579c2

Browse files
TechassiTechassi
committed
chore: Initial commit
0 parents  commit b4579c2

File tree

5 files changed

+621
-0
lines changed

5 files changed

+621
-0
lines changed

‎README.md

Lines changed: 203 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,203 @@
1+
# Actions
2+
3+
This repository contains various reusable actions which encapsulate series of
4+
commands to run a particular step in a workflow.
5+
6+
## Definitions
7+
8+
| Name | Example |
9+
| ---------------------------------- | ---------------------------------------------------------------------- |
10+
| Image Registry | `docker.stackable.tech` |
11+
| Image Repository | `stackable/kafka` |
12+
| Image Index Manifest Tag | `3.4.1-stackable0.0.0-dev` |
13+
| Image Manifest Tag | `3.4.1-stackable0.0.0-dev-amd64` |
14+
| Image Repository URI | `docker.stackable.tech/stackable/kafka` |
15+
| Image Index URI (if multi-arch) | `docker.stackable.tech/stackable/kafka:3.4.1-stackable0.0.0-dev` |
16+
| Image Manifest URI (if multi-arch) | `docker.stackable.tech/stackable/kafka:3.4.1-stackable0.0.0-dev-amd64` |
17+
| Image Repo Digest | `docker.stackable.tech/stackable/kafka@sha256:917f800259ef4915f976...` |
18+
| Digest | `sha256:917f800259ef4915f976e93987b752fd64debf347568610d7f685d2022...` |
19+
20+
## `build-product-image`
21+
22+
Manifest: [build-product-image/action.yml][build-product-image]
23+
24+
> [!NOTE]
25+
> The build step is not concerned with registries, ports, paths to repositories,
26+
> but still requires a name. If the name does not contain a registry,
27+
> `hub.docker.com` (?) is implied. Therefore, `localhost` will be used as the
28+
> registry so as to avoid accidental interactions with an unintended registry.
29+
>
30+
> Ideally, bake should be refactored to use `localhost` as the registry for the
31+
> previously mentioned reason (whether or not that is behind some option).
32+
33+
This action builds a *single* container image using `bake`. It does the
34+
following work:
35+
36+
1. Free disk space to avoid running out of disk space during larger builds.
37+
2. Build the image using `bake` which internally uses `docker buildx`.
38+
3. Temporarily retag the image to use `localhost` instead of
39+
`docker.stackable.tech/stackable`.
40+
4. Produce output values to be used in later steps.
41+
42+
This action is considered to be the **single** source of truth regarding image
43+
index tag and image manifest tag. All subsequent tasks must use these values to
44+
ensure consistency.
45+
46+
Currently, bake provides the following ouput in the `bake-target-tags` file:
47+
48+
```plain
49+
docker.stackable.tech/stackable/kafka:3.4.1-stackable0.0.0-dev-amd64
50+
```
51+
52+
Until bake supports the ability to specify the registry, this action will retag
53+
the image as:
54+
55+
```plain
56+
localhost/kafka:3.4.1-stackable0.0.0-dev-amd64
57+
```
58+
59+
### Inputs and Outputs
60+
61+
> [!TIP]
62+
> For descriptions of the inputs and outputs, see the [build-product-image]
63+
> workflow.
64+
65+
#### Inputs
66+
67+
- `product-name`
68+
- `product-version`
69+
- `image-tools-version`
70+
- `build-cache-username`
71+
- `build-cache-password`
72+
73+
#### Outputs
74+
75+
- `image-manifest-tag`
76+
77+
[build-product-image]: ./build-product-image/action.yml
78+
79+
## `publish-image`
80+
81+
Manifest: [publish-image/action.yml][publish-image]
82+
83+
This action signs and publishes a *single* container image to the given
84+
registry. It does the following work:
85+
86+
1. Tag the `source-image-uri` with the specified `image-registry-uti`,
87+
`image-repository`, and `image-repository`.
88+
2. Push the container image to the specified registry.
89+
3. Sign the container image (which pushes the signature to the specified
90+
registry).
91+
4. Generate an SBOM via a syft scan.
92+
5. Attest an image with the SBOM as a predicate (which pushes the attestation
93+
to the specified registry).
94+
95+
### Inputs and Outputs
96+
97+
> [!TIP]
98+
> For descriptions of the inputs and outputs, see the [publish-image] workflow.
99+
100+
<!-- markdownlint-disable-next-line MD028 -->
101+
> [!IMPORTANT]
102+
> For multi-arch images, the `image-manifest-tag` should have the `-$ARCH`
103+
> suffix, as the tag without it should be reserved for the image index manifest
104+
> which will refer to container images for each architecture we will push images
105+
> for.
106+
107+
#### Inputs
108+
109+
- `image-registry-uri`
110+
- `image-registry-username`
111+
- `image-registry-password`
112+
- `image-repository`
113+
- `image-manifest-tag`
114+
- `source-image-uri`
115+
116+
#### Outputs
117+
118+
None
119+
120+
[publish-image]: ./publish-image/action.yml
121+
122+
## `publish-index-manifest`
123+
124+
Manifest: [publish-index-manifest/action.yml][publish-index-manifest]
125+
126+
This action creates an image index manifest, publishes it, and signs it. It does
127+
the following work:
128+
129+
1. Create an image index manifest and link to each architecture in
130+
`image-architectures`.
131+
2. Push the image index manifest.
132+
3. Sign the image index manifest (which pushes the signature to the specified
133+
registry).
134+
135+
### Inputs and Outputs
136+
137+
> [!TIP]
138+
> For descriptions of the inputs and outputs, see the [publish-index-manifest]
139+
> workflow.
140+
141+
#### Inputs
142+
143+
- `image-registry-uri`
144+
- `image-registry-username`
145+
- `image-registry-password`
146+
- `image-repository`
147+
- `image-index-manifest-tag`
148+
- `image-architectures`
149+
150+
#### Outputs
151+
152+
None
153+
154+
[publish-index-manifest]: ./publish-index-manifest/action.yml
155+
156+
## `shard`
157+
158+
Manifest: [shard/action.yml][shard]
159+
160+
This action produces a list of versions for a product. This is to be used as a
161+
matrix dimension to parallelize builds. It does the following work:
162+
163+
1. Reads the `conf.py`, filtering versions for the product
164+
2. Write the JSON array of version to `$GITHUB_OUTPUT` for use in a matrix.
165+
166+
Example usage:
167+
168+
```yaml
169+
jobs:
170+
generate_matrix:
171+
name: Generate Version List
172+
runs-on: ubuntu-latest
173+
steps:
174+
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
175+
- id: shard
176+
uses: ./.github/actions/shard
177+
with:
178+
product-name: ${{ env.PRODUCT_NAME }}
179+
outputs:
180+
versions: ${{ steps.shard.outputs.versions }}
181+
182+
actual_matrix:
183+
needs: [generate_matrix]
184+
strategy:
185+
matrix:
186+
versions: ${{ fromJson(needs.generate_matrix.outputs.versions) }}
187+
# ...
188+
```
189+
190+
### Inputs and Outputs
191+
192+
> [!TIP]
193+
> For descriptions of the inputs and outputs, see the [shard] workflow.
194+
195+
#### Inputs
196+
197+
- `product-name`
198+
199+
#### Outputs
200+
201+
- `versions`
202+
203+
[shard]: ./publish-index-manifest/action.yml

‎build-product-image/action.yml

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
---
2+
name: Build Product Image
3+
description: This action builds a product Docker image with a specific version
4+
inputs:
5+
product-name:
6+
description: The name of the product to build via bake (directory name)
7+
required: true
8+
product-version:
9+
description: The version of the product to build via bake
10+
required: true
11+
image-tools-version:
12+
description: The Stackable image-tools version
13+
default: 0.0.12
14+
build-cache-username:
15+
description: Build cache username
16+
default: github
17+
build-cache-password:
18+
description: Build cache password
19+
required: true
20+
outputs:
21+
image-manifest-tag:
22+
description: |
23+
Human-readable tag (usually the version) with architecture information,
24+
for example: `3.4.1-stackable0.0.0-dev-amd64`
25+
value: ${{ steps.image_info.outputs.IMAGE_MANIFEST_TAG }}
26+
runs:
27+
using: composite
28+
steps:
29+
- name: Free Disk Space (Ubuntu)
30+
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # v1.3.1
31+
with:
32+
# This might remove tools that are actually needed, if set to "true" but
33+
# frees about 6 GB.
34+
tool-cache: false
35+
36+
# All of these default to true, but feel free to set to "false" if
37+
# necessary for your workflow.
38+
android: true
39+
dotnet: true
40+
haskell: true
41+
large-packages: true
42+
docker-images: true
43+
swap-storage: true
44+
45+
- name: Setup Docker Buildx
46+
uses: docker/setup-buildx-action@988b5a0280414f521da01fcc63a27aeeb4b104db # v3.6.1
47+
48+
# NOTE (@Techassi): Why do we install python via apt and not the setup-python action?
49+
- name: Setup Python
50+
shell: bash
51+
run: |
52+
set -euo pipefail
53+
sudo apt update
54+
sudo apt install -y python3
55+
56+
- name: Building ${{ inputs.product-name }}
57+
shell: bash
58+
run: echo ${{ inputs.product-name }}
59+
60+
- name: Install image-tools-stackabletech
61+
shell: bash
62+
run: pip install image-tools-stackabletech==${{ inputs.image-tools-version }}
63+
64+
# Needed if you pass the --cache argument to the bake command below
65+
- name: Login to the docker build cache registry
66+
uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567 # v3.3.0
67+
with:
68+
registry: build-repo.stackable.tech:8083
69+
username: ${{ inputs.build-cache-username }}
70+
password: ${{ inputs.build-cache-password }}
71+
72+
- name: Build image using bake
73+
env:
74+
IMAGE_REPOSITORY: ${{ inputs.product-name }}
75+
BAKE_PRODUCT_VERSION: ${{ inputs.product-version }}
76+
shell: bash
77+
run: |
78+
set -euo pipefail
79+
IMAGE_ARCH="$(uname -m | sed -e 's#x86_64#amd64#' | sed -e 's#aarch64#arm64#')"
80+
81+
bake \
82+
--product $IMAGE_REPOSITORY=$BAKE_PRODUCT_VERSION \
83+
--image-version "0.0.0-dev-${IMAGE_ARCH}" \
84+
--architecture "linux/${IMAGE_ARCH}" \
85+
--export-tags-file bake-target-tags \
86+
--cache
87+
88+
- name: Re-tag Image (Temporary)
89+
shell: bash
90+
run: |
91+
set -euo pipefail
92+
93+
# Extract the image uri and replace 'docker.stackable.tech/stackable'
94+
# with 'localhost' until bake does the right thing
95+
OLD_IMAGE_URI="$(< bake-target-tags)"
96+
97+
# Replace the image uri in the bake file
98+
sed -i -e 's/docker\.stackable\.tech\/stackable/localhost/' bake-target-tags
99+
100+
# Finally, re-tag image
101+
docker tag "$OLD_IMAGE_URI" "$(< bake-target-tags)"
102+
103+
- name: Extract Environment Variables
104+
id: image_info
105+
shell: bash
106+
run: |
107+
set -euo pipefail
108+
echo "bake-target-tags: "$(< bake-target-tags)
109+
110+
# Extract the image manifest tag from the bake-target-tags file
111+
IMAGE_MANIFEST_TAG=$(cut -d : -f 2 < bake-target-tags)
112+
[[ -n "$IMAGE_MANIFEST_TAG" ]]
113+
114+
# Add the contents of the env variables to the GitHub output, so that it
115+
# can be used as action outputs
116+
echo "IMAGE_MANIFEST_TAG=$IMAGE_MANIFEST_TAG" >> $GITHUB_OUTPUT

0 commit comments

Comments
 (0)