Is there an example about how to implement an external controller for AdmissionCheck

[Zhenshan-Jin]

How to define an external controller for AdmissionCheck? In the public documentation, I could see the built-in provisioning-request controller https://kueue.sigs.k8s.io/docs/concepts/admission_check/ but not the way how to build a custom one. (I am know to the k8s word so bare with me if this question is naive. ^ ^)

For example, I want to check in some records in the external database before the job is being admitted.

Any help is appreciated! If there is an complete example that would be perfect.

Thank you!

[Zhenshan-Jin]

hi @tenzen-y sorry to bother you. Could you help to shed some lights ? Thank you! ^ ^

[tenzen-y]

@Zhenshan-Jin Sorry for the late response. We have built-in AdmissionCheck Controller for ProvisioningRequest and MultiKueueCluster in the following:

• MultiKueueCluster: https://github.com/kubernetes-sigs/kueue/tree/c5fd605f15f882de9aff85dc2d7a24370241a4dc/pkg/controller/admissionchecks/multikueue
• ProvisioningRequest: https://github.com/kubernetes-sigs/kueue/tree/c5fd605f15f882de9aff85dc2d7a24370241a4dc/pkg/controller/admissionchecks/provisioning

But, unfortunately, we do not have any documentation to implement custom AdmissionCheck controller.
I will create a feature request issue so that we can expose the way for custom AdmissionCheck implementations.

[tenzen-y]

I created a feature request here: #2932

[mimowo]

We have a PR under review which aims to document the process: #2969. @Zhenshan-Jin feel free to comment / review on the PR to guide if there are some areas which you would like to be explained better.

[leipanhz]

@mimowo I have same question and try to study the example of ProvisioningRequest. My dev cluster is minikube, not sure if this is the reason,can set up ProvisioningRequest check:

{"level":"error","ts":"2024-10-07T17:30:36.914749619Z","logger":"setup","caller":"kueue/main.go:209","msg":"Provisioning Requests are not supported, skipped admission check controller setup","stacktrace":"main.setupIndexes\n\t/workspace/cmd/kueue/main.go:209\nmain.main\n\t/workspace/cmd/kueue/main.go:170\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:271"}

Can you suggest a fix or point me to a working example of external controller for Kueue?

[tenzen-y]

@leipanhz As @mimowo mentioned in #3195, could you check if ProvisioningRequest is enabled in your cluster?

[leipanhz]

@tenzen-y As replied in the other thread, I set up Provisioning in Kueue via https://github.com/kubernetes-sigs/kueue/blob/main/site/static/examples/provisioning/provisioning-setup.yaml. ProvisioningRequest is enabled by default from the documentation: https://kueue.sigs.k8s.io/docs/admission-check-controllers/provisioning Is there anythings to check?

[mimowo]

Replied in the thread.