Suggestions for k8s_onprem helm chart

[okyspace]

Hi, would like to suggest improvements to k8s_onprem helm chart.

Addons/changes for deploying triton in secure emvironment.

1. serviceaccount along with rbac to provide specific cluster rights for triton app.
2. secrets to store credentials to model repo like s3.
3. Configurable model repo instead of hardcoded nfs volume mount.
4. Configurable ingress True/False as there will be cases where ingress is not preferred.

Hope this is useful to make the helm chart more configurable for ease of future uogrades.

[dyastremsky]

CC: @Christina-Young-NVIDIA

[jharmison-redhat]

I'd like to revive this discussion on the direction of the Helm Charts, the primary deployment artifact for Triton on Kubernetes clusters.

I did already implement the ServiceAccount in #8005, for the k8s-onprem chart at least. There's more I'd like to be able to do.

Some examples of things I'm interested in:

• Stronger chart versioning to enable publishing them in a Helm repository with valid upgrade paths (maybe hosting community charts here on GitHub Pages?).
• The use of appVersion in Chart.yaml and a bit more templating regarding the image name, in line with the normal standards regarding image reference templating from parts.
• I'm +1 that NFS-only is pretty limiting for it to be hard coded in like it is.
• Introducing more configurable model repository storage options, and collapsing some of the other Charts down into a single chart that's configurable based on your IaaS/PaaS provider/storage environment. (e.g. NFS model repositories shouldn't only be accessible on prem, if it makes sense for a GCP-hosted inference server given a user's specific scenario)
• Moving some of the values from under the image key to an organization that better reflects how they're used (like numGpus as a starting example)

I'm able to help implement these things and chip in on reviews as well if the team is interested. I have a WIP branch set aside with some of the ideas implemented, but they were all kind of sweeping changes that depended pretty heavily on each other and I think it would be best to discuss them one at a time, implementing them bit by bit, to ensure that the Triton team is happy with the overall direction.

Are there any users of the Charts of Triton on Kubernetes with their own deployment artifacts who can chip in with some of their thoughts as well?

[dyastremsky]

CC: @statiraju @rmccorm4

[okyspace]

It will be good to include security configurations that can be enabled easily depending on deployment scenarios.