Way to specify order planner for steps

[lafriks]

My idea is to allow at pipeline level to specify:

order: sequential or order: parallel and maybe oven in future also to allow to specify order as an array of step names in what to execute or implement different step order planners

[lafriks]

Default would be order: auto - current that automatically selects either seq or parallel based on specified step options

[6543]

Looks to me lkie unecessary complexity. What usecase does this cover witch is not jet supported?

[lafriks]

Primary use case is to allow specify order: parallel without having to specify empty depends_on, others are just ideas how this theoretically could be extended in future if there is a new use case

[6543]

hmm I think depends_on: [""] could address that -> #3055 (comment)

witch would be addressed by #3008

[lafriks]

But adding empty depends_on imho looks more like a workaround than a clean solution if you want to execute all steps in parallel

[6543]

I think we can close this as answered as #3055 is fixed.

so it's adding depends_on: []

[qwerty287]

I'm still not convinced by this solution. Adding empty depends_on is very unintuitive. I'd prefer either using depends_on: clone as @xoxys suggested or adding another order as @lafriks suggested because both solution are much easier and clearer to read.

[xoxys]

Fully agree and I am confused now. As explained by @anbraten in the linked PR its not about an empty depends_on list but now this is the solution?

[qwerty287]

Yes, empty depends_on was always supposed to enable the dag, but since the yaml parser had the issue @anbraten fixed, this didn't work yet.

[lafriks]

This question was opened before we knew that empty depends_on does not work and my proposal was not to replace this way but add additional more cleaner way to specify this

[xoxys]

Am I the only one who thinks the new depends_on stuff for steps is a step backwards? Instead of just grouping steps together that should run in parallel, I now have to annotate every single step if I want some to run in parallel.

I know it allows some more complex dependency graphs but for simple cases where the only goal is to run 3 steps out of 10 in parallel depends_on makes it quite complicated.

From the docs: https://woodpecker-ci.org/docs/next/usage/workflow-syntax#depends_on

As soon as you set depends_on for a step a directed acyclic graph will be used and all steps of the workflow will be executed in parallel besides the steps that have a dependency set to another step using depends_on

That's why you have to annotate all steps to recreate a sequential order again for the remaining steps that should not run in parallel...

[xoxys]

I don't know if there's an easy way around this, because if I understand it correctly, this is simply how directed acyclic graphs work. However, from a user perspective, this behavior increases the complexity for simple environments.

[qwerty287]

I agree.

Maybe we could somehow solve this with something @lafriks suggeststed, i.e. to set a specific value to the (not yet existing) order key which then only uses parallel running if you use depends_on on the step.

Example:

order: sequence
steps:
  - name: step1
    ...
  - name: step2
    ...
  - name: step3
    depends_on: step1
    ...
  - name: step4
    ...

Would turn into stages: 1. step1 -> 2. step2 and step3 -> 3. step4.

(You could say: if a step does not have depends_on, automatically set depends_on to the previous step in yaml.)