Mega Feature: Support Pruning of resources

[barney-s]

Feature Description

Problem Statement:
If RGD is mutated to remove any resources then a previously applied instances would leak the removed resources. Eventually when an instance is deleted the leaked instances are not deleted as part of instance deletion.

Proposed Solution:
This is a common enough pattern in k8s ecosystem. kubectl attempts to solve this using applyset. There is a KEP describing the applyset solution. This has been adopted in the declarative pattern variant of kubebuilder.

We propose using applyset in KRO to handle tracking of resources that are part of a Instance. This allows us to handle pruning.

Alternatives Considered:
Rollout your own label matching logic and handle pruning.

Additional Context:

• Please vote on this issue by adding a 👍 reaction to the original issue
• If you are interested in working on this feature, please leave a comment

[greninja517]

This is one of the major issue that needs to be handled asap. I was thinking even implementing a simple deletion function before removing the instance finalizer could also help. We can utilize the labels already set by the controller to select out the orphaned resources and clean them up. But, orphaned resources will still remain in the cluster till the instance is deleted which is not a good practice though. However, it's still good to implement this rather than having nothing as of now. I am really interested in discussing about this feature. Looking forward to hear your thoughts !

[barney-s]

There is a PR (linked) out for this issue. Please provide your reviews.