-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Labels
Description
Problem
When PrefectDeployments run on Kubernetes work pools, they create Kubernetes Jobs that often fail due to insufficient RBAC permissions on the default service account.
Current Behavior
- PrefectDeployments create Kubernetes Jobs for flow execution
- These jobs run with the "default" service account in the deployment's namespace
- Jobs fail with errors like:
User "system:serviceaccount:my-namespace:default" cannot create resource "jobs" in API group "batch" in namespace "default"
Expected Behavior
The operator should automatically provision the necessary RBAC permissions when deploying Kubernetes work pools, so flow execution jobs can run successfully without manual RBAC configuration.
Current RBAC Setup
The operator itself has proper RBAC permissions for managing batch/jobs (deploy/charts/prefect-operator/templates/rbac.yaml:9-16
), but this doesn't extend to the service accounts used by flow execution jobs.
Suggested Implementation
When deploying a Kubernetes work pool, the operator should:
- Create a dedicated ServiceAccount for the work pool (or use a configurable one)
- Create appropriate ClusterRole/Role with permissions needed for flow execution:
batch/jobs
: create, get, list, watch, deletepods
: get, list, watch (for job monitoring)- Additional permissions as needed for flow requirements
- Create RoleBinding/ClusterRoleBinding connecting the ServiceAccount to the Role
- Configure the work pool to use this ServiceAccount for job execution
Alternative Approaches
- Allow users to specify an existing ServiceAccount in the work pool spec
- Provide documentation/examples for manual RBAC setup
- Create a separate CRD for managing flow execution permissions
Code References
- Operator RBAC:
deploy/charts/prefect-operator/templates/rbac.yaml
- Work pool controller:
internal/controller/prefectworkpool_controller.go
Related
This issue was extracted from the comprehensive issue report in #183.