Skip to content

Commit 5b66858

Browse files
feat: Add integration test action (#16)
* wip: initial implementation for replicated clusters * refactor: Move architecture extraction into script * chore: Install required tools (kuttl, stackablectl, beku) * chore: Remove > which apparently slipped through * chore: Replace -O with -o * refactor: Move tool directory setup into own step * chore: Install gettext-base package * feat: Run the integration test * chore: Add sudo to apt command * fix: Use correct context variable * chore: Make the comment a comment sigh... * fix: Prefix parameter with -- * fix: Remove superfluous quote * fix: Apply chmod +x * feat: Add start and end time outputs * feat: Use dynamic operator version based on branch * chore: chmod +x get_operator_version.sh * fix: Adjust quoting of shell commands * fix: Pass ref name to get_operator_version script * fix: Adjust outputs * fix: Add GH_TOKEN env var * feat: Add instance selection * fix: Export env vars * feat: Properly append test parameters * fix: Use correct env var * chore: Validate test parameters before cluster creation * chore: Add Kubernetes distribution tag * chore: Rework test parameter validation * chore: Add triggered-by tag * chore: Add README --------- Co-authored-by: Nick Larsen <nick.larsen@stackable.tech>
1 parent a3f7587 commit 5b66858

File tree

10 files changed

+304
-4
lines changed

10 files changed

+304
-4
lines changed

.scripts/get_architecture.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/usr/bin/env bash
2+
3+
uname -m | sed -e "s#x86_64#amd64#" | sed -e "s#aarch64#arm64#"

.scripts/get_operator_version.sh

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
#!/usr/bin/env bash
2+
set -uo pipefail
3+
4+
if [ "$1" == "main" ]; then
5+
echo "0.0.0-dev"
6+
exit
7+
fi
8+
9+
PR_NUMBER=$(gh pr view "$1" --json number --jq '.number')
10+
11+
if [ "$?" != "0" ]; then
12+
echo "0.0.0-dev"
13+
else
14+
echo "0.0.0-pr$PR_NUMBER"
15+
fi

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,5 +24,6 @@ particular step in a workflow.
2424
- [free-disk-space](./free-disk-space/README.md)
2525
- [publish-image](./publish-image/README.md)
2626
- [publish-index-manifest](./publish-index-manifest/README.md)
27+
- [run-integration-test](./run-integration-test/README.md)
2728
- [run-pre-commit](./run-pre-commit/README.md)
2829
- [shard](./shard/README.md)

build-container-image/action.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ runs:
6565
run: |
6666
set -euo pipefail
6767
68-
IMAGE_ARCH="$(uname -m | sed -e 's#x86_64#amd64#' | sed -e 's#aarch64#arm64#')"
68+
IMAGE_ARCH=$("$GITHUB_ACTION_PATH/../.scripts/get_architecture.sh")
6969
echo "IMAGE_ARCH=${IMAGE_ARCH}" | tee -a "$GITHUB_ENV"
7070
7171
IMAGE_MANIFEST_TAG="${IMAGE_INDEX_MANIFEST_TAG}-${IMAGE_ARCH}"

build-product-image/action.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ runs:
6565
shell: bash
6666
run: |
6767
set -euo pipefail
68-
IMAGE_ARCH="$(uname -m | sed -e 's#x86_64#amd64#' | sed -e 's#aarch64#arm64#')"
68+
IMAGE_ARCH=$("$GITHUB_ACTION_PATH/../.scripts/get_architecture.sh")
6969
7070
echo "::group::bake"
7171
bake \

publish-image/action.yml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,8 @@ runs:
8585
docker image push "$IMAGE_MANIFEST_URI"
8686
8787
# Output for the next step
88-
echo "IMAGE_REPO_DIGEST=$($GITHUB_ACTION_PATH/../.scripts/get_repo_digest.sh $IMAGE_MANIFEST_URI)" | tee -a "$GITHUB_ENV"
88+
IMAGE_REPO_DIGEST=$("$GITHUB_ACTION_PATH/../.scripts/get_repo_digest.sh" "$IMAGE_MANIFEST_URI")
89+
echo "IMAGE_REPO_DIGEST=$IMAGE_REPO_DIGEST" | tee -a "$GITHUB_ENV"
8990
9091
- name: Sign the container image (${{ env.IMAGE_REPO_DIGEST }})
9192
shell: bash

publish-index-manifest/action.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ runs:
9797
set -euo pipefail
9898
9999
# Get the image index manifest digest
100-
DIGEST=$($GITHUB_ACTION_PATH/../.scripts/get_manifest_digest.sh "$IMAGE_INDEX_URI")
100+
DIGEST=$("$GITHUB_ACTION_PATH/../.scripts/get_manifest_digest.sh" "$IMAGE_INDEX_URI")
101101
102102
# Construct the image repo digest, which for example contains:
103103
# docker.stackable.tech/stackable/kafka@sha256:91...

run-integration-test/README.md

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# `run-integration-test`
2+
3+
> Manifest: [run-integration-test/action.yml][run-integration-test]
4+
5+
This action runs an operator integration test. It does the following work:
6+
7+
1. Create a test cluster on-the-fly using the requested Kubernetes version and distribution via
8+
Replicated.
9+
2. Run the integration test based on the provided test parameters.
10+
3. Delete the cluster of the tests are done and send out a notification on failure.
11+
12+
## Inputs and Outputs
13+
14+
> [!TIP]
15+
> For descriptions of the inputs and outputs, see the complete [run-integration-test] action.
16+
17+
### Inputs
18+
19+
- `test-platform`(required, eg: `kind-1.31.0-amd64`)
20+
- `test-run` (required, `test-suite` or `test`)
21+
- `test-parameter` (defaults to `smoke`)
22+
- `replicated-api-token` (required)
23+
24+
### Outputs
25+
26+
- `start-time`
27+
- `end-time`
28+
29+
[run-integration-test]: ./action.yml

run-integration-test/action.yml

Lines changed: 216 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,216 @@
1+
---
2+
name: Run Integration Test
3+
description: |
4+
This action runs Stackable Operator integration tests on various platforms and
5+
Kubernetes distributions.
6+
inputs:
7+
test-platform:
8+
description: |
9+
The platform/distribution to run on (eg: `okd-4.15-amd64`)
10+
required: true
11+
test-run:
12+
description: Type of test run
13+
required: true
14+
test-parameter:
15+
description: Parameter to `--test-suite` or `--test` (ignored for `all`)
16+
default: ""
17+
replicated-api-token:
18+
description: Replicated API token (only needed if running on replicated)
19+
default: ""
20+
outputs:
21+
start-time:
22+
description: The date and time this integration test was started.
23+
value: ${{ steps.start-time.outputs.START_TIME }}
24+
end-time:
25+
description: The date and time this integration test finished.
26+
value: ${{ steps.end-time.outputs.END_TIME }}
27+
runs:
28+
using: composite
29+
steps:
30+
- name: Record Start Time
31+
id: start-time
32+
shell: bash
33+
run: |
34+
echo "START_TIME=$(date +'%Y-%m-%dT%H:%M:%S')" | tee -a "$GITHUB_OUTPUT"
35+
36+
- name: Extract Test and Instance Configuration
37+
env:
38+
TEST_PARAMETER: ${{ inputs.test-parameter }}
39+
TEST_PLATFORM: ${{ inputs.test-platform }}
40+
TEST_RUN: ${{ inputs.test-run }}
41+
shell: bash
42+
run: |
43+
set -euo pipefail
44+
45+
#####################################
46+
# Extract Kubernetes-related Values #
47+
#####################################
48+
49+
export KUBERNETES_DISTRIBUTION=$(echo "$TEST_PLATFORM" | cut -d - -f 1)
50+
export KUBERNETES_VERSION=$(echo "$TEST_PLATFORM" | cut -d - -f 2)
51+
export KUBERNETES_ARCHITECTURE=$(echo "$TEST_PLATFORM" | cut -d - -f 3)
52+
53+
echo "KUBERNETES_DISTRIBUTION=$KUBERNETES_DISTRIBUTION" | tee -a "$GITHUB_ENV"
54+
echo "KUBERNETES_VERSION=$KUBERNETES_VERSION" | tee -a "$GITHUB_ENV"
55+
echo "KUBERNETES_ARCHITECTURE=$KUBERNETES_ARCHITECTURE" | tee -a "$GITHUB_ENV"
56+
57+
##################################
58+
# Extract Instance Configuration #
59+
##################################
60+
61+
export INSTANCE_SIZE=$(yq '.instance-size' -e ./tests/infrastructure.yaml)
62+
INSTANCE_TYPE=$(yq '.[env(KUBERNETES_DISTRIBUTION)].[env(KUBERNETES_ARCHITECTURE)].[env(INSTANCE_SIZE)]' -e "$GITHUB_ACTION_PATH/instances.yml")
63+
64+
echo "INSTANCE_TYPE=$INSTANCE_TYPE" | tee -a "$GITHUB_ENV"
65+
66+
############################
67+
# Validate Test Parameters #
68+
############################
69+
70+
if [ -z "$TEST_RUN" ]; then
71+
echo "TEST_RUN must be defined and not empty"
72+
exit 1
73+
fi
74+
75+
if [ "$TEST_RUN" != "all" ]; then
76+
if [ -z "$TEST_PARAMETER" ]; then
77+
echo "TEST_PARAMETER must be defined and not empty"
78+
exit 1
79+
fi
80+
81+
if [ "$TEST_RUN" == "test-suite" ]; then
82+
yq '.suites[] | select(.name == env(TEST_PARAMETER))' -e ./tests/test-definition.yaml
83+
elif [ "$TEST_RUN" == "test" ]; then
84+
yq '.tests[] | select(.name == env(TEST_PARAMETER))' -e ./tests/test-definition.yaml
85+
fi
86+
fi
87+
88+
echo "TEST_PARAMETER=$TEST_PARAMETER" | tee -a "$GITHUB_ENV"
89+
echo "TEST_RUN=$TEST_RUN" | tee -a "$GITHUB_ENV"
90+
91+
- name: Prepare Replicated Cluster
92+
if: env.KUBERNETES_DISTRIBUTION != 'ionos'
93+
id: prepare-replicated-cluster
94+
uses: replicatedhq/replicated-actions/create-cluster@v1 # todo, hash
95+
with:
96+
# See: https://github.com/replicatedhq/replicated-actions/tree/main/create-cluster#inputs
97+
api-token: ${{ inputs.replicated-api-token }}
98+
cluster-name: integration-test-${{ github.repository }}-${{ github.run_id }}
99+
instance-type: ${{ env.INSTANCE_TYPE }}
100+
kubernetes-distribution: ${{ env.KUBERNETES_DISTRIBUTION }}
101+
kubernetes-version: ${{ env.KUBERNETES_VERSION }}
102+
ttl: 4h # todo: allow this to be configurable
103+
disk: 50 # todo: allow this to be configurable
104+
nodes: 1 # todo: allow this to be configurable
105+
tags: |
106+
- key: node-architecture
107+
value: ${{ env.KUBERNETES_ARCHITECTURE }}
108+
- key: kubernetes-distribution
109+
value: ${{ env.KUBERNETES_DISTRIBUTION }}
110+
- key: triggered-by
111+
value: ${{ github.triggering_actor }}
112+
113+
- name: Set Replicated kubeconfig
114+
if: env.KUBERNETES_DISTRIBUTION != 'ionos'
115+
env:
116+
KUBECONFIG: ${{ steps.prepare-replicated-cluster.outputs.cluster-kubeconfig }}
117+
shell: bash
118+
run: |
119+
set -euo pipefail
120+
mkdir ~/.kube
121+
echo "$KUBECONFIG" > ~/.kube/config
122+
123+
- name: Extract Operator Name
124+
env:
125+
REPOSITORY: ${{ github.repository }}
126+
shell: bash
127+
run: |
128+
set -euo pipefail
129+
130+
OPERATOR_NAME=$(echo "$REPOSITORY" | cut -d / -f 2 | sed 's/-operator//g')
131+
echo "OPERATOR_NAME=$OPERATOR_NAME" | tee -a "$GITHUB_ENV"
132+
133+
- name: Setup Tool Directory
134+
shell: bash
135+
run: |
136+
set -euo pipefail
137+
138+
TOOL_DIRECTORY="$HOME/.local/bin"
139+
mkdir -p "$TOOL_DIRECTORY"
140+
141+
echo "$TOOL_DIRECTORY" | tee -a "$GITHUB_PATH"
142+
echo "TOOL_DIRECTORY=$TOOL_DIRECTORY" | tee -a "$GITHUB_ENV"
143+
144+
# We don't need to install kubectl, kind or helm because it is already part of the installed
145+
# tools of the runner image.
146+
# See https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/install-kubernetes-tools.sh
147+
- name: Install kubectl-kuttl
148+
shell: bash
149+
run: |
150+
set -euo pipefail
151+
152+
curl -L -o "$TOOL_DIRECTORY/kubectl-kuttl" https://github.com/kudobuilder/kuttl/releases/download/v0.19.0/kubectl-kuttl_0.19.0_linux_x86_64
153+
chmod +x "$TOOL_DIRECTORY/kubectl-kuttl"
154+
155+
# Python3 is already installed, if we ever need to specify the version, we can use the
156+
# setup-python action.
157+
# See https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/install-python.sh
158+
- name: Install beku
159+
shell: bash
160+
run: |
161+
set -euo pipefail
162+
pip install beku-stackabletech
163+
164+
# mikefarah/yq is already installed on the runner
165+
# See https://github.com/actions/runner-images/blob/main/images/ubuntu/scripts/build/install-yq.sh
166+
167+
- name: Install stackablectl
168+
shell: bash
169+
run: |
170+
set -euo pipefail
171+
172+
curl -L -o "$TOOL_DIRECTORY/stackablectl" https://github.com/stackabletech/stackable-cockpit/releases/latest/download/stackablectl-x86_64-unknown-linux-gnu
173+
chmod +x "$TOOL_DIRECTORY/stackablectl"
174+
175+
- name: Install apt packages
176+
shell: bash
177+
run: |
178+
set -euo pipefail
179+
180+
sudo apt update
181+
sudo apt install -y \
182+
gettext-base
183+
184+
- name: Run Integration Test (${{ inputs.test-run }}=${{ inputs.test-parameter }})
185+
env:
186+
REF_NAME: ${{ github.ref_name }}
187+
GH_TOKEN: ${{ github.token }}
188+
shell: bash
189+
run: |
190+
set -euo pipefail
191+
192+
OPERATOR_VERSION=$("$GITHUB_ACTION_PATH/../.scripts/get_operator_version.sh" "$REF_NAME")
193+
python ./scripts/run-tests --skip-tests --operator "$OPERATOR_NAME=$OPERATOR_VERSION"
194+
195+
if [ "$TEST_RUN" == "all" ]; then
196+
python ./scripts/run-tests --skip-release --log-level debug
197+
else
198+
python ./scripts/run-tests --skip-release --log-level debug "--$TEST_RUN" "$TEST_PARAMETER"
199+
fi
200+
201+
- name: Destroy Replicated Cluster
202+
if: env.KUBERNETES_DISTRIBUTION != 'ionos' && always()
203+
# If the creation of the cluster failed, we don't want to error and abort
204+
continue-on-error: true
205+
uses: replicatedhq/replicated-actions/remove-cluster@v1 # todo, hash
206+
with:
207+
# See: https://github.com/replicatedhq/replicated-actions/tree/main/remove-cluster#inputs
208+
api-token: ${{ inputs.replicated-api-token }}
209+
cluster-id: ${{ steps.prepare-replicated-cluster.outputs.cluster-id }}
210+
211+
- name: Record End Time
212+
id: end-time
213+
if: always()
214+
shell: bash
215+
run: |
216+
echo "END_TIME=$(date +'%Y-%m-%dT%H:%M:%S')" | tee -a "$GITHUB_OUTPUT"

run-integration-test/instances.yml

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
eks:
2+
arm64:
3+
small: m7g.large
4+
medium: m7g.2xlarge
5+
large: m7g.4xlarge
6+
amd64:
7+
small: m6i.large
8+
medium: m6i.2xlarge
9+
large: m6i.4xlarge
10+
11+
gke:
12+
arm64:
13+
small: t2a-standard-2
14+
medium: t2a-standard-8
15+
large: t2a-standard-16
16+
amd64:
17+
small: e2-standard-2
18+
medium: e2-standard-8
19+
large: e2-standard-16
20+
21+
aks:
22+
arm64:
23+
small: Standard_D2ps_v5
24+
medium: Standard_D8ps_v5
25+
large: Standard_D16ps_v5
26+
amd64:
27+
small: Standard_DS1_v2
28+
medium: Standard_DS3_v2
29+
large: Standard_DS5_v2
30+
31+
kind:
32+
amd64:
33+
small: r1.small
34+
medium: r1.medium
35+
large: r1.large

0 commit comments

Comments
 (0)