[BUG] [Project Monitoring V2] Rhel 8.6 observed project registration namespace did not recreate when deleted

[ronhorton]

Rancher Server Setup

• Rancher version: v2.6.7-rc3
• Installation option (Docker install/Helm Chart): Helm HA
  • If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): rke1 custom rhel 8.6
• Proxy/Cert Details:

Information about the Cluster

• Kubernetes version: 1.24
• Cluster Type (Local/Downstream): custom downstream provisioned with RHEL 8.6: ami-053835e36b16f97d0
  • If downstream, what type of cluster? (Custom/Imported or specify provider for Hosted/Infrastructure Provider):

User Information

• What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom)
  • If custom, define the set of permissions: admin

Describe the bug

After installing prometheus-federator, deleted the project registration namespace in the default project. The project registration namespace was not recreated as expected. After redeploying the prom-federator pod, the registration namespace is recreated.

To Reproduce

1. installed prom fed
2. clicked cluster > projects & namespaces
3. right clicked the 3 dots for the project registration namespace
4. delete

Result
namespace is not recreated
Expected Result
namespace is recreated

Screenshots

Additional context

`apiVersion: management.cattle.io/v3
kind: Project
metadata:
annotations:
authz.management.cattle.io/creator-role-bindings: '{"created":["project-owner"],"required":["project-owner"]}'
field.cattle.io/creatorId: user-9wkck
lifecycle.cattle.io/create.mgmt-project-rbac-remove: "true"
lifecycle.cattle.io/create.project-namespace-auth_c-hn9db: "true"
creationTimestamp: "2022-07-25T19:57:36Z"
finalizers:

• controller.cattle.io/mgmt-project-rbac-remove
• clusterscoped.controller.cattle.io/project-namespace-auth_c-hn9db
  generateName: p-
  generation: 4
  labels:
  authz.management.cattle.io/default-project: "true"
  cattle.io/creator: norman
  managedFields:
• apiVersion: management.cattle.io/v3
  fieldsType: FieldsV1
  fieldsV1:
  f:metadata:
  f:annotations:
  .: {}
  f:authz.management.cattle.io/creator-role-bindings: {}
  f:field.cattle.io/creatorId: {}
  f:lifecycle.cattle.io/create.mgmt-project-rbac-remove: {}
  f:lifecycle.cattle.io/create.project-namespace-auth_c-hn9db: {}
  f:finalizers:
  .: {}
  v:"clusterscoped.controller.cattle.io/project-namespace-auth_c-hn9db": {}
  v:"controller.cattle.io/mgmt-project-rbac-remove": {}
  f:generateName: {}
  f:labels:
  .: {}
  f:authz.management.cattle.io/default-project: {}
  f:cattle.io/creator: {}
  f:spec:
  .: {}
  f:clusterName: {}
  f:description: {}
  f:displayName: {}
  f:enableProjectMonitoring: {}
  f:status:
  .: {}
  f:conditions: {}
  f:podSecurityPolicyTemplateId: {}
  manager: rancher
  operation: Update
  time: "2022-07-25T20:08:30Z"
  name: p-rwrjs
  namespace: c-hn9db
  resourceVersion: "32004"
  uid: 7ece94f0-b881-46a6-a3ef-ae8c67aaa9c7
  spec:
  clusterName: c-hn9db
  description: Default project created for the cluster
  displayName: Default
  enableProjectMonitoring: false
  status:
  conditions:
• lastUpdateTime: "2022-07-25T19:57:36Z"
  status: "True"
  type: BackingNamespaceCreated
• lastUpdateTime: "2022-07-25T19:57:36Z"
  status: "True"
  type: CreatorMadeOwner
• lastUpdateTime: "2022-07-25T19:57:36Z"
  status: "True"
  type: InitialRolesPopulated
• lastUpdateTime: "2022-07-25T20:08:30Z"
  status: "True"
  type: DefaultNamespacesAssigned
  podSecurityPolicyTemplateId: ""`