Skip to content

Commit ae3d165

Browse files
adwk67razvan
adwk67
and
razvan
authored
bundles script (#46)
Co-authored-by: Razvan-Daniel Mihai <84674+razvan@users.noreply.github.com>
1 parent 705501d commit ae3d165

File tree

7 files changed

+221
-320
lines changed

7 files changed

+221
-320
lines changed

‎olm/README.md

Lines changed: 65 additions & 98 deletions
Original file line numberDiff line numberDiff line change
@@ -1,123 +1,90 @@
1-
# OLM installation files
1+
# Overview
22

3-
The following steps describe how to install a Stackable operator - in this, the operator for Apache Zookeeper - using the [Operator Lifecycle Manager](https://olm.operatorframework.io/) (OLM).
3+
These notes are a summary of the more in-depth, internal documentation [here](https://app.nuclino.com/Stackable/Engineering/Certification-Process-a8cf57d0-bd41-4d56-b505-f59af4159a56).
44

5-
It specifically installs the version 23.1.0 of the operator. Installing additional versions in the future requires generating new bundle images and updating the catalog as described below.
5+
# Prerequisites
66

7-
## Usage
8-
9-
Prerequisite is of course a running OpenShift cluster.
10-
11-
First, install the operator using OLM:
12-
13-
kubectl apply -f catalog-source.yaml \
14-
-f operator-group.yaml \
15-
-f subscription.yaml
16-
17-
Then, install the operator dependencies with Helm:
18-
19-
helm install secret-operator stackable/secret-operator
20-
helm install commons-operator stackable/commons-operator
21-
22-
And finally, create an Apache Zookeeper cluster:
23-
24-
kubectl create -f examples/simple-zookeeper-cluster.yaml
25-
26-
NOTE: The `kuttl` tests don't work because they themselves require SCCs which are not available.
27-
28-
## OLM packaging requirements
29-
30-
- An [OpenShift](https://developers.redhat.com/products/openshift-local/overview) cluster.
7+
- an [OpenShift](https://developers.redhat.com/products/openshift-local/overview) cluster with the `stackable-operators` namespace
318
- [opm](https://github.com/operator-framework/operator-registry/)
329
- docker and kubectl
33-
- `kubeadmin` access
34-
35-
It was tested with:
36-
37-
$ crc version
38-
WARN A new version (2.5.1) has been published on https://developers.redhat.com/content-gateway/file/pub/openshift-v4/clients/crc/2.5.1/crc-linux-amd64.tar.xz
39-
CRC version: 2.4.1+b877358
40-
OpenShift version: 4.10.14
41-
Podman version: 4.0.2
42-
43-
$ oc version
44-
Client Version: 4.10.14
45-
Server Version: 4.10.14
46-
Kubernetes Version: v1.23.5+b463d71
47-
48-
$ opm version
49-
Version: version.Version{OpmVersion:"v1.23.2", GitCommit:"82505333", BuildDate:"2022-07-04T13:45:39Z", GoOs:"linux", GoArch:"amd64"}
50-
51-
## Open questions
52-
53-
- OLM [doesn't support DaemonSet(s)](https://github.com/operator-framework/operator-lifecycle-manager/issues/1022) and we need them for the secret-operator. Currently we can deploy the secret-operator using Helm but this means we cannot configure the [required](https://olm.operatorframework.io/docs/tasks/creating-operator-manifests/#required-apis) apis of the Zookeeper bundle. What are the consequences for publishing and certification ?
54-
- Here we create a catalog for a single operator. We probably want a catalog for all Stackable operators in the future but this will get large very quickly. Figure out how to handle this. Especially figure out what happens with new versions of the same operator.
55-
- OLM cannot create SecurityContextConstraints objects. The Zookeeper cluster (not the operator) cannot run with the default `restricted` SCC. The current solution is to use the `hostmount-anyuid` SCC for the `zookeeper-clusterrole`. Will this pass the certification process ?
56-
- Everything (catalog, subscription, etc) is installed in the `stackable-operators` namespace. Is this a good idea ?
57-
- The Subscription object uses `installPlanApproval: Automatic` which means the operator is updated automatically for every new version. Is this a good idea?
58-
59-
See the [OLM documentation](https://olm.operatorframework.io/docs/tasks/) for details.
60-
61-
## Build and publish operator bundle image
62-
63-
Each catalog can contain several operator packages, and each operator package can contain multiple channels, each with its own bundles of different versions of the operator.
64-
65-
### Generate operator bundle (this is operator-specific)
66-
67-
opm alpha bundle generate --directory manifests --package zookeeper-operator-package --output-dir bundle --channels stable --default stable
10+
- `kubeadmin` access: once logged in to an openshift cluster a secret is needed
11+
```
12+
export KUBECONFIG=~/.kube/config
13+
oc create secret generic kubeconfig --from-file=kubeconfig=$KUBECONFIG --namespace stackable-operators
14+
```
15+
- [tkn](https://github.com/tektoncd/cli)
16+
- a secret for `pyxis_api_key`: a token giving access to Redhat Connect
17+
- a secret for `github-api-token`: a token giving access to the RH repo
6818

69-
### Build bundle image
19+
# Deployment
7020

71-
docker build -t docker.stackable.tech/stackable/zookeeper-operator-bundle:23.1.0 -f bundle.Dockerfile .
72-
docker push docker.stackable.tech/stackable/zookeeper-operator-bundle:23.1.0
21+
Stackable operators can be deployed to Openshift in one of three ways:
7322

74-
### Validate bundle image
23+
- Helm: either natively, or by using stackablectl
24+
- Operator Catalog
25+
- Certified Operators
7526

76-
opm alpha bundle validate --tag docker.stackable.tech/stackable/zookeeper-operator-bundle:23.1.0 --image-builder docker
27+
The latter two require an operator to be deployed to an Openshift cluster, from where the operator (and its dependencies, if these have been defined) can be installed from the console UI. Both pathways use version-specific manifest files which are created in the [Stackable repository](https://github.com/stackabletech/openshift-certified-operators) that is forked from [here](https://github.com/redhat-openshift-ecosystem/certified-operators). These manifests are largely based on the templates used by helm, with the addition of Openshift-specific items (such as a ClusterServiceVersion manifest).
7728

78-
## Create catalog
29+
## Build the bundle
7930

80-
mkdir catalog
81-
opm generate dockerfile catalog
31+
An operator bundle and catalog can be built and deployed using the `build-bundle.sh` script e.g.
8232

83-
## Create a package for each operator
33+
```
34+
./olm/build-bundles.sh -r 23.4.1 -b secret-23.4.1 -o secret-operator -d
35+
```
8436

85-
opm init zookeeper-operator-package \
86-
--default-channel=stable \
87-
--description=./README.md \
88-
--output yaml > catalog/zookeeper-operator-package.yaml
37+
Where:
38+
- `-r <release>`: the release number (mandatory). This must be a semver-compatible value to patch-level e.g. 23.1.0.
39+
- `-b <branch>`: the branch name (mandatory) in the (stackable forked) openshift-certified-operators repository.
40+
- `-o <operator-name>`: the operator name (mandatory) e.g. airflow-operator.
41+
- `-d <deploy>`: optional flag for catalog deployment.
8942

90-
{
91-
echo "---"
92-
echo "schema: olm.channel"
93-
echo "package: zookeeper-operator-package"
94-
echo "name: stable"
95-
echo "entries:"
96-
echo "- name: zookeeper-operator.v23.1.0"
97-
} >> catalog/zookeeper-operator-package.yaml
43+
The script creates a catalog specific to an operator. A catalog can contain bundles for multiple operators, but a 1:1 deployment makes it easier to deploy and test operators independently. Testing with a deployed catalog is essential as the certification pipeline should only be used for stable operators, and a certified operator can only be changed if a new version is specified.
9844

99-
NOTE: with the command below we can add the Stackable logo as icon.
45+
## Use the CI pipeline
10046

101-
# add for each operator...
102-
opm render docker.stackable.tech/stackable/zookeeper-operator-bundle:23.1.0 --output=yaml >> catalog/zookeeper-operator-package.yaml
47+
### Testing
10348

104-
# ...and then validate the entire catalog
105-
opm validate catalog
49+
This should either be called from the stackable-utils root folder, or the `volumeClaimTemplateFile` path should be changed accordingly.
10650

107-
The catalog is correct if the command above returns successfully without any message. If the catalog doesn't validate, the operator will not install. Now build a catalog image and push it to the repository:
51+
```
52+
export GIT_REPO_URL=https://github.com/stackabletech/openshift-certified-operators
53+
export BUNDLE_PATH=operators/stackable-commons-operator/23.4.1
54+
export LOCAL_BRANCH=commons-23.4.1
10855
109-
docker build . -f catalog.Dockerfile -t docker.stackable.tech/stackable/zookeeper-operator-catalog:latest
110-
docker push docker.stackable.tech/stackable/zookeeper-operator-catalog:latest
56+
tkn pipeline start operator-ci-pipeline \
57+
--namespace stackable-operators \
58+
--param git_repo_url=$GIT_REPO_URL \
59+
--param git_branch=$LOCAL_BRANCH \
60+
--param bundle_path=$BUNDLE_PATH \
61+
--param env=prod \
62+
--param kubeconfig_secret_name=kubeconfig \
63+
--workspace name=pipeline,volumeClaimTemplateFile=olm/templates/workspace-template.yml \
64+
--showlog
65+
```
11166

112-
## Install catalog and the operator group
67+
### Certifying
11368

114-
kubectl apply -f catalog-source.yaml
115-
kubectl apply -f operator-group.yaml
69+
This callout is identical to the previous one, with the addition of the last two parameters `upstream_repo_name` and `submit=true`:
11670

117-
## List available operators
71+
```
72+
export GIT_REPO_URL=https://github.com/stackabletech/openshift-certified-operators
73+
export BUNDLE_PATH=operators/stackable-commons-operator/23.4.1
74+
export LOCAL_BRANCH=commons-23.4.1
11875
119-
kubectl get packagemanifest -n stackable-operators
76+
tkn pipeline start operator-ci-pipeline \
77+
--namespace stackable-operators \
78+
--param git_repo_url=$GIT_REPO_URL \
79+
--param git_branch=$LOCAL_BRANCH \
80+
--param bundle_path=$BUNDLE_PATH \
81+
--param env=prod \
82+
--param kubeconfig_secret_name=kubeconfig \
83+
--workspace name=pipeline,volumeClaimTemplateFile=olm/templates/workspace-template.yml \
84+
--showlog \
85+
--param upstream_repo_name=redhat-openshift-ecosystem/certified-operators \
86+
--param submit=true
87+
```
12088

121-
## Install operator
89+
A successful callout will result in a PR being opened in the Redhat repository, from where progress can be tracked.
12290

123-
kubectl apply -f subscription.yaml

‎olm/build-bundles.sh

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
#!/usr/bin/env bash
2+
# Usage:
3+
# ./olm/build-bundles.sh -r <release as x.y.z> -b <branch-name>
4+
# -r <release>: the release number (mandatory). This must be a semver-compatible value to patch-level e.g. 23.1.0.
5+
# -b <branch>: the branch name (mandatory) in the (stackable forked) openshift-certified-operators repository.
6+
# -o <operator-name>: the operator name (mandatory) e.g. airflow-operator.
7+
# -d <deploy>: optional flag for catalog deployment.
8+
#
9+
# e.g. ./olm/build-bundles.sh -r 23.4.1 -b secret-23.4.1 -o secret-operator -d
10+
11+
set -euo pipefail
12+
set -x
13+
14+
15+
SCRIPT_NAME=$(basename $0)
16+
17+
parse_inputs() {
18+
INITIAL_DIR="$PWD"
19+
20+
VERSION=""
21+
BRANCH=""
22+
OPERATOR_NAME=""
23+
DEPLOY=false
24+
25+
while [[ "$#" -gt 0 ]]; do
26+
case $1 in
27+
-r|--release) VERSION="$2"; shift ;;
28+
-b|--branch) BRANCH="$2"; shift ;;
29+
-o|--operator) OPERATOR_NAME="$2"; shift ;;
30+
-d|--deploy) DEPLOY=true ;;
31+
*) echo "Unknown parameter passed: $1"; exit 1 ;;
32+
esac
33+
shift
34+
done
35+
36+
# e.g. "airflow" instead of "airflow-operator"
37+
OPERATOR=$(echo "${OPERATOR_NAME}" | cut -d- -f1)
38+
}
39+
40+
bundle-clean() {
41+
rm -rf "bundle"
42+
rm -rf "bundle.Dockerfile"
43+
}
44+
45+
build-bundle() {
46+
opm alpha bundle generate --directory manifests --package "${OPERATOR_NAME}-package" --output-dir bundle --channels stable --default stable
47+
cp metadata/*.yaml bundle/metadata/
48+
docker build -t "docker.stackable.tech/stackable/${OPERATOR_NAME}-bundle:${VERSION}" -f bundle.Dockerfile .
49+
docker push "docker.stackable.tech/stackable/${OPERATOR_NAME}-bundle:${VERSION}"
50+
opm alpha bundle validate --tag "docker.stackable.tech/stackable/${OPERATOR_NAME}-bundle:${VERSION}" --image-builder docker
51+
}
52+
53+
setup() {
54+
if [ -d "catalog" ]; then
55+
rm -rf catalog
56+
fi
57+
58+
mkdir -p catalog
59+
rm -f catalog.Dockerfile
60+
rm -f catalog-source.yaml
61+
}
62+
63+
catalog() {
64+
opm generate dockerfile catalog
65+
66+
echo "Initiating package: ${OPERATOR}"
67+
opm init "stackable-${OPERATOR}-operator" \
68+
--default-channel=stable \
69+
--description=./README.md \
70+
--output yaml > "catalog/stackable-${OPERATOR}-operator.yaml"
71+
echo "Add operator to package: ${OPERATOR}"
72+
{
73+
echo "---"
74+
echo "schema: olm.channel"
75+
echo "package: stackable-${OPERATOR}-operator"
76+
echo "name: stable"
77+
echo "entries:"
78+
echo "- name: ${OPERATOR}-operator.v${VERSION}"
79+
} >> "catalog/stackable-${OPERATOR}-operator.yaml"
80+
echo "Render operator: ${OPERATOR}"
81+
opm render "docker.stackable.tech/stackable/${OPERATOR}-operator-bundle:${VERSION}" --output=yaml >> "catalog/stackable-${OPERATOR}-operator.yaml"
82+
83+
echo "Validating catalog..."
84+
opm validate catalog
85+
86+
echo "Build and push catalog for all ${OPERATOR} operator..."
87+
docker build . -f catalog.Dockerfile -t "docker.stackable.tech/stackable/stackable-${OPERATOR}-catalog:${VERSION}"
88+
docker push "docker.stackable.tech/stackable/stackable-${OPERATOR}-catalog:${VERSION}"
89+
}
90+
91+
deploy() {
92+
if $DEPLOY; then
93+
echo "Deploying catalog..."
94+
95+
{
96+
echo "---"
97+
echo "apiVersion: operators.coreos.com/v1alpha1"
98+
echo "kind: CatalogSource"
99+
echo "metadata:"
100+
echo " name: stackable-${OPERATOR}-catalog"
101+
echo " namespace: stackable-operators"
102+
echo "spec:"
103+
echo " sourceType: grpc"
104+
echo " image: docker.stackable.tech/stackable/stackable-${OPERATOR}-catalog:${VERSION}"
105+
echo " displayName: Stackable Catalog"
106+
echo " publisher: Stackable GmbH"
107+
echo " updateStrategy:"
108+
echo " registryPoll:"
109+
echo " interval: 10m"
110+
} >> catalog-source.yaml
111+
112+
kubectl apply -f catalog-source.yaml
113+
echo "Catalog deployment successful!"
114+
fi
115+
}
116+
117+
main() {
118+
parse_inputs "$@"
119+
if [ -z "${VERSION}" ] || [ -z "${BRANCH}" ] || [ -z "${OPERATOR_NAME}" ]; then
120+
echo "Usage: $SCRIPT_NAME -r <release> -b <branch> -o <operator>"
121+
exit 1
122+
fi
123+
124+
TMPFOLDER=$(mktemp -d -t 'openshift-bundles-XXXXXXXX')
125+
cd "${TMPFOLDER}"
126+
127+
git clone "git@github.com:stackabletech/openshift-certified-operators.git" --depth 1 --branch "${BRANCH}" --single-branch "${TMPFOLDER}/openshift-certified-operators/"
128+
129+
cd "${TMPFOLDER}/openshift-certified-operators/operators/stackable-${OPERATOR}-operator/${VERSION}"
130+
131+
# clean up any residual files from previous actions
132+
bundle-clean
133+
build-bundle
134+
135+
# should not be pushed to repo (unintentionally) once bundle is built, so clean up straight away
136+
bundle-clean
137+
138+
echo "Bundle-build successful!"
139+
140+
pushd "$INITIAL_DIR/olm"
141+
setup
142+
catalog
143+
deploy
144+
145+
popd
146+
echo "Catalog built successfully!"
147+
}
148+
149+
main "$@"

0 commit comments

Comments
 (0)