Skip to content

fix: E2E Security demo - Separate DB setup from superset resource #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions stacks/end-to-end-security/rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: superset-job-rb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: superset-job-role
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:serviceaccounts
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: superset-job-role
rules:
- apiGroups:
- batch
resources:
- jobs
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- statefulsets
verbs:
- get
- list
- watch
50 changes: 50 additions & 0 deletions stacks/end-to-end-security/setup-postgresql.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
---
apiVersion: batch/v1
kind: Job
metadata:
name: setup-db-job
spec:
template:
spec:
initContainers:
# The postgres image does not contain curl or wget...
- name: download-dump
image: oci.stackable.tech/sdp/testing-tools:0.2.0-stackable0.0.0-dev
command:
- bash
- -c
- |
kubectl rollout status --watch statefulset/postgresql-superset
cd /tmp
curl --fail -O https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/postgres_superset_dump.sql.gz
gunzip postgres_superset_dump.sql.gz

# We need to omit changing the users password, as otherwise the content in the Secrets does not match
# the actual password in Postgres.
grep -vwE '(CREATE ROLE postgres;|CREATE ROLE superset;|ALTER ROLE postgres|ALTER ROLE superset)' postgres_superset_dump.sql > /dump/postgres_superset_dump.sql
volumeMounts:
- name: dump
mountPath: /dump/
containers:
- name: restore-postgres
image: docker.io/bitnami/postgresql:16.1.0-debian-11-r11 # Same image as the bitnami postgres helm-chart is using
command:
- bash
- -c
- |
echo "Preparing restore..."
psql --host postgresql-superset --user postgres < /dump/postgres_superset_dump.sql
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: postgresql-superset
key: postgres-password
volumeMounts:
- name: dump
mountPath: /dump/
volumes:
- name: dump
emptyDir: {}
restartPolicy: OnFailure
backoffLimit: 20
39 changes: 2 additions & 37 deletions stacks/end-to-end-security/superset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,49 +21,14 @@ spec:
replicas: 1
podOverrides:
spec:
# We need to restore the postgres state before the superset container itself starts some database migrations
initContainers:
# The postgres image does not contain curl or wget...
- name: download-dump
- name: wait-for-setup-db-job
image: oci.stackable.tech/sdp/testing-tools:0.2.0-stackable0.0.0-dev
command:
- bash
- -c
- |
cd /tmp
curl --fail -O https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/postgres_superset_dump.sql.gz
gunzip postgres_superset_dump.sql.gz

# We need to omit changing the users password, as otherwise the content in the Secrets does not match
# the actual password in Postgres.
grep -vwE '(CREATE ROLE postgres;|CREATE ROLE superset;|ALTER ROLE postgres|ALTER ROLE superset)' postgres_superset_dump.sql > /dump/postgres_superset_dump.sql
volumeMounts:
- name: dump
mountPath: /dump/
- name: restore-postgres
image: docker.io/bitnami/postgresql:16.1.0-debian-11-r11 # Same image as the bitnami postgres helm-chart is using
command:
- bash
- -c
- |
if psql --host postgresql-superset --user postgres --csv -c "SELECT datname FROM pg_database where datname = 'superset' limit 1" | grep -q superset; then
# The flask app will do any necessary migrations.
echo "Skip restoring the DB as it already exists"
exit 0
fi
psql --host postgresql-superset --user postgres < /dump/postgres_superset_dump.sql
env:
- name: PGPASSWORD
valueFrom:
secretKeyRef:
name: postgresql-superset
key: postgres-password
volumeMounts:
- name: dump
mountPath: /dump/
volumes:
- name: dump
emptyDir: {}
kubectl wait --for=condition=complete job/setup-db-job
---
apiVersion: v1
kind: Secret
Expand Down
2 changes: 2 additions & 0 deletions stacks/stacks-v2.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -582,6 +582,8 @@ stacks:
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/trino.yaml
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/trino-regorules.yaml
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/trino-policies.yaml
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/rbac.yaml
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/setup-postgresql.yaml
- plainYaml: https://raw.githubusercontent.com/stackabletech/demos/main/stacks/end-to-end-security/superset.yaml
parameters:
- name: keycloakAdminPassword
Expand Down