💡PROPOSAL: Automate generation and injection of the webhook related certificates

[p-strusiewiczsurmacki-mobica]

What

As per #313 - currently Coil (by default) uses certificates generated with make cert (which uses internal tool) for webhooks, or user has to incorporate other cert-generation tools (like. e.g. cert-manager). We would like to streamline this process by incorporating open-policy-agent/cert-controller into Coil.

How

Cert-controller is based on controller-runtime - it can be added to the controller-runtime's manager and started with mgr.Start().

The workflow look as follows:

1. Add cert-controller to the manager.
2. Move all manager-related setup logic to goroutine that waits for cert-controller to finish certificate generation and injection.
3. Start manager in goroutine so it won't block the main thread.
4. Watch for both of the goroutines completion/errors.

The general plan for implementation would be:

• 1. Add cert-manager to coil-ipam-controller and coil-egress-controller (both have webhooks).
• 2. Change customization files to not use webhook injection patches as default.
• 3. Decide to what to do with v2/cmd/gencert and add relevant changes.
• 4. Update tests - as certificates are generated and injected after Coil's start there might be a delay between the start and reaching the full operability state, therefore tests should reflect that.
• 5. Update relevant docs - e.g. quick start guide, e2e guide etc.

Checklist

• Finish implentation of the issue
• Test all functions
• Have enough logs to trace activities
• Notify developers of necessary actions

[p-strusiewiczsurmacki-mobica]

As I believe that it might be better to have some code earlier than later, so you can test the approach, I have also created initial PR #319 which implements required changes as marked above.

[chez-shanpu]

@p-strusiewiczsurmacki-mobica Thank you for your proposal and PR!
I think this function would be useful for users because it reduces dependence on other tools as written in this issue and in-cluster tools like cert-manager cannot be deployed before Coil is started up (since Pods' IP cannot be assigned when we run Coil with IPAM mode).

At first glance, the workflow seems good :) (btw cert-manager in the second step is actually cert-controller?)
I'll check and comment about the codes in the PR you made.

About v2/cmd/gencert in the implementation plan, I think it should be kept for backward compatibility.

[p-strusiewiczsurmacki-mobica]

btw cert-manager in the second step is actually cert-controller?

Yes, it is. :) Changed that. :)

[chez-shanpu]

Completed #319