RFC: one Pod per Workflow (K8s)

[zc-devs]

Follow up of #3345

The "one-pod-many-containers" solution is by far the best solution for the underlying problems, and also comes with the benefit of less time spent between the pod creations

Running every containers on the same pod and FWIW that is how Tekton worked.

Drone uses a single pod approach and that transitioning from Drone to WP would be more seamless if WP would do the same.
It simplifies caching and removes the need for a RWX/volume mount in the helm chart (as a pipeline could also just operate on emptyDir then within a pod).
Downside of that approach would be that resource-based scheduling would only be possible on the pipeline level and not step level, but I personally would see that rather as an improvement as it avoids having to define backend_options for every step.

There seems to be a lot of advantages to that approach. However to me it also sounds like a much more complex change. I'm not sure if is is already something that is on the roadmap of the project and universally agreed upon.
Unless the maintainers are committed to change the architecture from one-pod-per-step to one-pod-per-workflow (or are confident that this change can be made reasonably soon)...

[zc-devs]

It simplifies caching

What caching are you talking about? Cache between steps or between runs?
One Pod per Workflow is not related to later. Some external systems resolve it, S3 or special tools.
Do someone need cache between steps?
We are talking about some build process and caching its dependencies, compilation artifacts, right?

[zc-devs]

compilation artifacts

I use it as well as Woodpecker project. Source is compiled to binary, then in next step binary is packaged to docker image.
But I would not call it caching. Also Volume serves me well in that case.

[pat-s]

I referred to keeping the source files between steps, yes. Which is currently done via a volume mounted to each step. It is not caching in a direct way, you're right. And nothing which is not already existing.

And yes, "real" cache would need to be stored externally like you mentioned, S3 or similar and require an external plugin to up & download.

I mingled up some points/terminology here, apologies.

[pat-s]

For ref, here are other discussions which discussed this topic already:

• Ability to configure the type of the volume for the agent in the kubernetes backend #1594
• clone step using RWX volume takes long time to complete helm#142