Oracle Cloud LoadBalancer Registrar

Kubernetes Operator for Oracle Cloud Infrastructure (OCI) that automatically registers Kubernetes worker nodes to Oracle Cloud LoadBalancers. This operator monitors Kubernetes nodes and dynamically manages their registration with OCI Load Balancers (both Classic and Network Load Balancers).

Overview

This operator provides the following features:

• Automatic node registration: Automatically registers newly added nodes in the Kubernetes cluster to the backend set of an Oracle Cloud Infrastructure (OCI) Load Balancer.
• Dynamic management: Continuously monitors node additions and removals and updates the Load Balancer configuration accordingly.
• Service integration: Automatically detects the NodePort of a Kubernetes Service resource and uses it to register the node with the Load Balancer.
• Multi-service support: Single LBRegistrar resource can manage multiple services and backend sets, eliminating conflicts.
• Multi-Load Balancer support: Supports multiple LBRegistrar resources to register nodes with different Load Balancers.

Usage

1. Setup OCI LoadBalancer: Create a LoadBalancer in OCI Console or using OCI CLI. Make sure to create a backend set for the LoadBalancer.

2. Create NodePort Service: Create a NodePort service in Kubernetes to expose the application.

3. Create OCI user and API key: Create a user and API key in OCI Console or using OCI CLI. This user will be used to authenticate to update the LoadBalancer.

4. OCI API Key secret creation:

   kubectl -n key-namespace create secret generic oci-api-key \
   --from-file=private-key=path/to/your/oci_api_key.pem

5. Deploy Operator:

   kubectl apply -f dist/install.yaml

6. Create LBRegistrar resource:

   Single Service (Legacy):

   apiVersion: nodes.peppy-ratio.dev/v1alpha1
   kind: LBRegistrar
   metadata:
     name: my-app-registrar
   spec:
     loadBalancerId: "ocid1.loadbalancer.oc1.ap-tokyo-1.xxxxx"
     backendSetName: "my-backend-set"
     weight: 1
     apiKey:
       user: "ocid1.user.oc1..xxxxx"
       fingerprint: "aa:bb:cc:dd:ee"
       tenancy: "ocid1.tenancy.oc1..xxxxx"
       region: "ap-tokyo-1"
       privateKey:
         namespace: key-namespace
         secretKeyRef:
           name: oci-api-key
           key: private-key
     service:
       name: "my-nodeport-service"
       namespace: "default"
       port: http2
       filterByEndpoints: true

   Multiple Services (Recommended):

   apiVersion: nodes.peppy-ratio.dev/v1alpha1
   kind: LBRegistrar
   metadata:
     name: my-app-multiservice-registrar
   spec:
     loadBalancerId: "ocid1.loadbalancer.oc1.ap-tokyo-1.xxxxx"
     apiKey:
       user: "ocid1.user.oc1..xxxxx"
       fingerprint: "aa:bb:cc:dd:ee"
       tenancy: "ocid1.tenancy.oc1..xxxxx"
       region: "ap-tokyo-1"
       privateKey:
         namespace: key-namespace
         secretKeyRef:
           name: oci-api-key
           key: private-key
     services:
       - name: "my-nodeport-service"
         namespace: "default"
         port: https
         backendSetName: "https-backend"
         weight: 1
         filterByEndpoints: true
       - name: "my-nodeport-service"
         namespace: "default"
         port: http2
         backendSetName: "http-backend"
         weight: 1
         filterByEndpoints: true

Working Principle

1. When LBRegistrar resource is created, the operator establishes a connection to the OCI LoadBalancer
2. The operator monitors all nodes in the Kubernetes cluster
3. When a new node is added, it automatically registers the node with the LoadBalancer using the specified NodePort
4. When a node is removed, it automatically removes the node from the LoadBalancer
5. If Service resources are specified, the operator dynamically retrieves the NodePort from each Service
6. For multi-service configurations, all services are processed sequentially within a single reconciliation cycle to avoid conflicts
7. WorkRequest Management: Each backend update waits for OCI WorkRequest completion to ensure no state conflicts

Multi-Service Benefits

The new multi-service support eliminates the OCI Load Balancer conflict issues that occurred when multiple LBRegistrar resources targeted the same load balancer:

• Conflict Resolution: No more "Invalid State Transition" errors from concurrent updates
• Simplified Management: Single resource manages multiple backend sets
• Atomic Operations: All backend updates happen within one reconciliation cycle
• WorkRequest Synchronization: Waits for each OCI operation to complete before proceeding to the next
• Backward Compatibility: Existing single-service configurations continue to work unchanged

Reliability Features

WorkRequest Management

The controller implements robust WorkRequest handling for OCI operations:

• Completion Waiting: Each UpdateBackendSet operation waits for WorkRequest completion (up to 5 minutes)
• Status Monitoring: Polls WorkRequest status every 5 seconds with detailed logging
• Error Handling: Properly handles Failed, Canceled, and timeout scenarios
• Network Load Balancer Support: Full WorkRequest implementation for NLB operations

LBRegistrar Spec

• LBRegistrarSpec

  • loadBalancerId: OCI Load Balancer OCID.
  • nodePort: Target port on nodes. Ignored when service or services is used.
  • port: Deprecated alias for nodePort.
  • service: Single service to fetch nodePort. Deprecated by services.
  • services: List of services for multi-service registration.
  • weight: Weight of backends.
  • backendSetName: Name of the backend set.
  • apiKey: API key configuration.

• ServiceSpec

  • name: Target Service name.
  • namespace: Service namespace.
  • port: Service port (number or name).
  • filterByEndpoints: Register only nodes with matching endpoints.
  • weight: Weight for this service's backends.
  • backendSetName: Backend set specific to this service.

• ApiKeySpec

  • user: OCI user OCID.
  • fingerprint: API key fingerprint.
  • tenancy: Tenancy OCID.
  • region: OCI region.
  • privateKey: Reference to secret holding the private key.

• PrivateKeySpec

  • namespace: Namespace of the secret.
  • secretKeyRef: Secret key selector.

• LBRegistrarStatus

  • phase: Current lifecycle phase.

See the docs directory for design details.

License

This project is licensed under the GNU General Public License v3.0 - see the LICENSE file for details.