feat(operator): migrate from kube-rbac-proxy to built-in controller-runtime protection

[iamjoemccormick]

Resolves #35

This drops the kube-rbac-proxy sidecar in favor of controller-runtime's native WithAuthenticationAndAuthorization filter, introduced in v0.18+.This change is required because the gcr.io/kubebuilder/kube-rbac-proxy image has been deprecated and is scheduled for removal: kubernetes-sigs/kubebuilder#3907

I choose not to upgrade to the latest kubebuilder release because this requires re-scaffolding and reintegrating our custom code (which is fairly extensive). Instead I followed the steps in the FAQ "How can I manually change my project to switch to Controller-Runtime's built-in auth protection?".

Testing

// Setup Minikube, install OLM and BeeGFS requirements:
$ minikube start --kubernetes-version='1.32.3'
$ curl -L https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.25.0/install.sh -o install.sh
$ chmod +x install.sh
$ ./install.sh v0.25.0
$ hack/minikube_install_driver_prerequisites.sh

// Build/deploy the operator:
$ export IMAGE_TAG_BASE="ghcr.io/iamjoemccormick/test-beegfs-csi-driver-operator"
$ export VERSION=1.8.0-beta.1
$ make build docker-build docker-push
$ make manifests bundle bundle-build bundle-push
$ operator-sdk run bundle $IMAGE_TAG_BASE-bundle:v$VERSION

// Test the metrics endpoint:
$ kubectl create clusterrolebinding beegfs-csi-driver-operator-metrics-binding \
  --clusterrole=beegfs-csi-driver-operator-metrics-reader \
  --serviceaccount=default:beegfs-csi-driver-operator-controller-manager
$ kubectl create token beegfs-csi-driver-operator-controller-manager
<TOKEN>

$ kubectl run curl-test --rm -i --tty --image=curlimages/curl -- /bin/sh
$ curl -k https://beegfs-csi-driver-operator-controller-manager-metrics-service.default.svc.cluster.local:8443/metrics
Unauthorized
$ export TOKEN="<TOKEN>"
$ curl -k -H "Authorization: Bearer $TOKEN" https://beegfs-csi-driver-operator-controller-manager-metrics-service.default.svc.cluster.local:8443/metrics
# HELP certwatcher_read_certificate_errors_total Total number of certificate read errors
# TYPE certwatcher_read_certificate_errors_total counter
certwatcher_read_certificate_errors_total 0
# HELP certwatcher_read_certificate_total Total number of certificate reads
# TYPE certwatcher_read_certificate_total counter
[...]

[iamjoemccormick]

Verified upgrading from a previous release is not broken by this change. After the upgrade the kube-rbac-proxy container is no longer present:

$ export IMAGE_TAG_BASE=ghcr.io/iamjoemccormick/test-beegfs-csi-driver-operator
$ kubectl create ns beegfs-csi
$ operator-sdk run bundle $IMAGE_TAG_BASE-bundle:v1.7.0-rc0 -n beegfs-csi
INFO[0010] Creating a File-Based Catalog of the bundle "ghcr.io/iamjoemccormick/test-beegfs-csi-driver-operator-bundle:v1.7.0-rc0" 
INFO[0011] Generated a valid File-Based Catalog         
INFO[0017] Created registry pod: mjoemccormick-test-beegfs-csi-driver-operator-bundle-v1-7-0-rc0 
INFO[0017] Created CatalogSource: beegfs-csi-driver-operator-catalog 
INFO[0017] OperatorGroup "operator-sdk-og" created      
INFO[0017] Created Subscription: beegfs-csi-driver-operator-v1-7-0-rc0-sub 
INFO[0025] Approved InstallPlan install-lchhp for the Subscription: beegfs-csi-driver-operator-v1-7-0-rc0-sub 
INFO[0025] Waiting for ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.7.0-rc0" to reach 'Succeeded' phase 
INFO[0025]   Waiting for ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.7.0-rc0" to appear 
INFO[0028]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.7.0-rc0" phase: Pending 
INFO[0030]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.7.0-rc0" phase: Installing 
INFO[0048]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.7.0-rc0" phase: Succeeded 
INFO[0048] OLM has successfully installed "beegfs-csi-driver-operator.v1.7.0-rc0"
$ operator-sdk run bundle-upgrade $IMAGE_TAG_BASE-bundle:v1.8.0-beta.1 -n beegfs-csi
INFO[0001] Found existing subscription with name beegfs-csi-driver-operator-v1-7-0-rc0-sub and namespace beegfs-csi 
INFO[0001] Found existing catalog source with name beegfs-csi-driver-operator-catalog and namespace beegfs-csi 
INFO[0011] Generated a valid Upgraded File-Based Catalog 
INFO[0013] Created registry pod: emccormick-test-beegfs-csi-driver-operator-bundle-v1-8-0-beta-1 
INFO[0013] Updated catalog source beegfs-csi-driver-operator-catalog with address and annotations 
INFO[0013] Deleted previous registry pod with name "mjoemccormick-test-beegfs-csi-driver-operator-bundle-v1-7-0-rc0" 
INFO[0023] Approved InstallPlan install-tzvd9 for the Subscription: beegfs-csi-driver-operator-v1-7-0-rc0-sub 
INFO[0023] Waiting for ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.8.0-beta.1" to reach 'Succeeded' phase 
INFO[0023]   Waiting for ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.8.0-beta.1" to appear 
INFO[0025]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.8.0-beta.1" phase: Pending 
INFO[0027]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.8.0-beta.1" phase: Installing 
INFO[0043]   Found ClusterServiceVersion "beegfs-csi/beegfs-csi-driver-operator.v1.8.0-beta.1" phase: Succeeded 
INFO[0043] Successfully upgraded to "beegfs-csi-driver-operator.v1.8.0-beta.1"
$ kubectl get csv -n beegfs-csi
NAME                                       DISPLAY             VERSION        REPLACES                                PHASE
beegfs-csi-driver-operator.v1.8.0-beta.1   BeeGFS CSI Driver   1.8.0-beta.1   beegfs-csi-driver-operator.v1.7.0-rc0   Succeeded
$ $ kubectl get pod beegfs-csi-driver-operator-controller-manager-778f9cf645-grxww -n beegfs-csi \
  -o jsonpath='{.spec.containers[*].name}'
manager