Nil pointer dereference in ShardingDatabaseReconciler causes controller manager to crash

[mwapededalus]

Hi all,

When deploying the Oracle Database Operator in Kubernetes, the controller manager pods experience a nil pointer dereference error in the ShardingDatabaseReconciler component. This causes pods to crash and enter a CrashLoopBackOff state, preventing the operator from functioning properly.

Error:

E0424 12:58:27.629355 1 panic.go:262] "Observed a panic" panic="runtime error: invalid memory address or nil pointer dereference" panicGoValue="\"invalid memory address or nil pointer dereference\"" stacktrace=<goroutine 668 [running]:k8s.io/apimachinery/pkg/util/runtime.logPanic({0x2f80748, 0x45dcfa0}, {0x25978e0, 0x457aac0}) /root/go/pkg/mod/k8s.io/apimachinery@v0.31.3/pkg/util/runtime/runtime.go:107 +0xbc k8s.io/apimachinery/pkg/util/runtime.handleCrash({0x2f80748, 0x45dcfa0}, {0x25978e0, 0x457aac0}, {0x45dcfa0, 0x0, 0x43aaa5?}) /root/go/pkg/mod/k8s.io/apimachinery@v0.31.3/pkg/util/runtime/runtime.go:82 +0x5e k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc000afcfc0?}) /root/go/pkg/mod/k8s.io/apimachinery@v0.31.3/pkg/util/runtime/runtime.go:59 +0x108 panic({0x25978e0?, 0x457aac0?}) /usr/local/go/src/runtime/panic.go:785 +0x132 github.com/oracle/oracle-database-operator/controllers/database.(*ShardingDatabaseReconciler).SetupWithManager.(*ShardingDatabaseReconciler).eventFilterPredicate.func2({{0x2faaf78?, 0xc001d7d680?}, {0x2faaf78?, 0xc0022f3400?}}) /workspace/controllers/database/shardingdatabase_controller.go:462 +0xb5

Environment:

Azure Kubernetes Service
Oracle Database Operator version: Tested with main, 1.2.0, 1.1.0, 1.0.0
Standard deployment using the official manifest

How to reproduce bug:

• Create a namespace for the operator: kubectl create namespace oracle-operator-system
• Download the operator manifest: curl -o oracle-database-operator.yaml https://raw.githubusercontent.com/oracle/oracle-database-operator/main/oracle-database-operator.yaml
• Apply the manifest: kubectl apply -f oracle-database-operator.yaml -n oracle-operator-system
• Observe the controller manager pods: kubectl get pods -n oracle-operator-system
• Check the logs of the crashing pods: kubectl logs -n oracle-operator-system

I've also tried reducing the replica count from 3 to 1. Error still persists.

Any pointers would be greatly appreciated.

[NikPerne]

Hi, I also have this issue... I was trying to upgrade the operator from 1.1.0 to 1.2.0 and when the operator pods were created everytime one of the pods was elected as leader this error occured.

[hussain2]

I am facing the same issue on AKS k8s version v1.29.15 with cilium cni.
I applied operator yaml file for v1.2.0.
Operator is in crash restart loop.