-
Notifications
You must be signed in to change notification settings - Fork 1.6k
✨ Allow helm plugin to use custom prefixes #5119
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
✨ Allow helm plugin to use custom prefixes #5119
Conversation
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: liam-mackie The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Welcome @liam-mackie! |
Hi @liam-mackie. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
@Kavinjsir / @varshaprasad96 - is there any possibility of getting this one reviewed/approved to test? I'd love to get this in! |
pkg/plugins/optional/helm/v2alpha/scaffolds/internal/kustomize/helm_templater_test.go
Show resolved
Hide resolved
fs.BoolVar(&p.force, "force", false, "if true, regenerates all the files") | ||
fs.StringVar(&p.manifestsFile, "manifests", DefaultManifestsFile, | ||
"path to the YAML file containing Kubernetes manifests from kustomize output") | ||
fs.StringVar(&p.kustomizePath, "kustomize-path", DefaultKustomizeFile, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not think that we should pass the path here, otherwise we will need a flag for each file.
In this case, we might want to get the config from PROJECT file which has the project name instead of check the kustomize file wdyt?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately, there's no reference to the entrypoint for Kustomize in the PROJECT file, and there's no way to infer the prefix someone uses in Kustomize outside of the kustomization.yaml
itself. Kubebuilder conventions put it in config/default/kustomization.yaml
. Originally, I left this as unconfigurable, and simply assumed that the kustomization.yaml
would always be in this path.
The best we have to get this programatically is the Makefile, which uses it in the build-installer
target:
$(KUSTOMIZE) build config/default > dist/install.yaml
The config/default
path is what we'd use, and we'd simply look for a kustomization.yaml
in there. I'm not sure if that's any better than simply allowing the desired kustomization file to be passed through though.
Happy to defer to your expertise here!
|
||
// ServiceMonitor scaffolds a ServiceMonitor for Prometheus monitoring in the Helm chart | ||
type ServiceMonitor struct { | ||
machinery.TemplateMixin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The best solution would be to have the ServiceMonitor from the kustomize install.yaml
Could we look at this option?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When prometheus is enabled in kustomize, it'll use the ServiceMonitor from the kustomize install.yaml - this code path is only called when prometheus is disabled in kustomize.
I assume it's to allow prometheus to be enabled/disabled in the values.yaml of the helm chart.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that is the reason for we template this one.
But I think we can look for:
name: {{ .NamePrefix }}-controller-manager-metrics-monitor
serverName: {{ .NamePrefix }}-controller-manager-metrics-service
Those values in the install.yaml and use here instead of use PROJECT name.
Example:
- The serverName is the name of the service so we can look for: https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v4-with-plugins/dist/install.yaml#L922-L939
- And use the prefix found above for the name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah ok, I think I understand - so take the actual values of the service name from the install.yaml rather than assuming it's the project name or prefix?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the clarification - I'll get something up ASAP for you to take a look at 🙇
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But then we need to remove the rest only get the data from the install.yaml
We should avoid adding flags since it will make it harder to keep things maintained
Could we keep the solution here only looking for the install.yaml to get this value?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, I meant that it's an option that we can consider, rather than an actual option that the user provides! It's not a new flag, and is the only way the data will be passed in to this template!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It will be a new flag for the plugin: fs.StringVar(&p.kustomizePath, "kustomize-path", DefaultKustomizeFile,
That means specific tests for it, and also if we pass one file, we might need to pass others, and it can grow, etc. Also, users will need to pass more than one value by default to properly generate it and bring a bad UX
So, the best approach is we get the values from the install.yaml
We should not consume any other place.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see - you mean the kustomize path. I'll get something working and run it by you, I have an idea here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Get data from kustomize output rather than kustomization.yaml changes the way we get the prefix to instead get it from the deployment name. Is this closer to what you were hoping for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your contribution 🥇
I tried to give the first round of reviews. I think we need to see if we could use the data from install.yaml and check for more generic options. Please, check my comments , let me know wdyt
Background:
When working with long names, you will often need to use a shortened (or abbreviated) version of the controller name as the
namePrefix
in thekustomization.yaml
file. This means a shorter name can be used as the prefix to all the files.When using the
v2alpha/helm
plugin, this mostly works. The only resource that doesn't use the name generated by Kustomize is theServiceMonitor
. This is problematic, as it will use the name (with the custom prefix), then template the chart name (usually the long name) in front of it. This leads to a broken installation.The fix:
This PR fixes this by attempting to get the
namePrefix
from the kustomization config path. If thenamePrefix
and theprojectName
are different, we do not template the file for the service monitor. This way, customisations are honoured, but existing behaviour is maintained.