[SURE-8849] fleet incorrectly reports pre-existing resources as missing when setting helm.takeOwnership: true

[jhoblitt]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

When settings helm.takeOwernship: true , the chart deployment will add the standard helm labels and annotations to an existing object. E.g.:

apiVersion: ceph.rook.io/v1
kind: CephFilesystem
metadata:
  annotations:
    meta.helm.sh/release-name: rook-ceph-conf
    meta.helm.sh/release-namespace: rook-ceph
    ...
  labels:
    app.kubernetes.io/managed-by: Helm

However, fleet-agent will complain about missing resources:

time="2024-02-07T23:16:25Z" level=error msg="bundle kueyen-only-kueyen-fleet-rook-ceph-conf: cephfilesystem.ceph.rook.io rook-ceph/obs-env missing"

As will the bundle and bundleDeployment resources:

NAME                                      DEPLOYED   MONITORED   STATUS
kueyen-only-kueyen-fleet-rook-ceph-conf   True       True        cephfilesystem.ceph.rook.io rook-ceph/obs-env missing

NAME                                      BUNDLEDEPLOYMENTS-READY   STATUS
kueyen-only-kueyen-fleet-rook-ceph-conf   0/1                       Modified(1) [Cluster fleet-default/kueyen]; cephfilesystem.ceph.rook.io rook-ceph/obs-env missing; cephnfs.ceph.rook.io rook-ceph/obs-env missing; cephobjectstore.ceph.rook.io rook-ceph/lfa missing

Expected Behavior

I expect that "fleet" takes ownership of existing resources and does not report them as missing (when they exists!).

The resource is reported as missing because the pre-existing resources do not have the objectset.rio.cattle.io/hash label and objectset.rio.cattle.io/id annotation set. E.g.:

    annotations:
      objectset.rio.cattle.io/id: default-kueyen-only-kueyen-fleet-rook-ceph-conf
      ...
    labels:
      app.kubernetes.io/managed-by: Helm
      objectset.rio.cattle.io/hash: 563cb3553e5fba84eee5ba3054b9b537c653465a
     ...

Manually copying the value of objectset.rio.cattle.io/hash from another resource resolves the "missing" resource warning.

Steps To Reproduce

1. Manually create a resource
2. Create a fleet bundle with helm.takeOwnership: true with a duplicate of that resource
3. Observe warnings on the bundle, bundleDeployment resources and in the fleet-agent logs.

Environment

- Architecture: x86-64
- Fleet Version: 0.9.0 (rancher 2.8.1)
- Cluster:
  - Provider: RKE1
  - Kubernetes Version: v1.25.9

Logs

No response

Anything else?

No response

[jhoblitt]

It looks like this issue comes up when transferring a resource between bundles. The label/annotations are not updated.

[jhoblitt]

It seems that the annotation isn't required to resolve the missing error (at least in some cases). Setting the objectset.rio.cattle.io/hash label alone is sufficient.

[lindhe]

Setting the objectset.rio.cattle.io/hash label alone is sufficient.

How does one do that? I've gotten stuck with a similar issue with Fleet reporting resources as "Missing" while they do actually exist. :(

EDIT: I was able to grep for the hash and id using kubectl grep RoleTemplate -o yaml | grep objectset and then set it for my missing resources using kubectl label and kubectl annotate.