'workflow-of-workflows' where inner workflow uses parameters gets error on submit

[rupello]

I'm trying to understand the workflow-of-workflows example to see if I can solve a problem I have with my workflow getting too large.

I modified the example by using the diamond-DAG as the inner workflow, but get an error on submitting this

Failed to submit workflow: templates.main.steps[0].sub-workflow-1-0 templates.w1-resource: failed to resolve {{inputs.parameters.message}}

I noticed that the generated file is missing single quotes in the embedded workflow resource: - {{inputs.parameters.message}} instead of - '{{inputs.parameters.message}}'

not sure if this is a bug or something I am doing the wrong way?

    from hera.workflows import DAG, Container, Parameter, Workflow, Resource, Steps, Step

    with Workflow(
        generate_name="inner-workflow-",
        entrypoint="diamond",
    ) as w_inner:
        echo = Container(
            name="echo",
            image="alpine:3.7",
            command=["echo", "{{inputs.parameters.message}}"],
            inputs=[Parameter(name="message")],
        )
        with DAG(name="diamond"):
            A = echo(name="A", arguments={"message": "A"})
            B = echo(name="B", arguments={"message": "B"})
            C = echo(name="C", arguments={"message": "C"})
            D = echo(name="D", arguments={"message": "D"})
            A >> [B, C] >> D
            
    with Workflow(generate_name="outer-workflow-", 
                    entrypoint="main", 
                    service_account_name='worker') as w_outer:
        w1_resource = Resource(
            name="w1-resource",
            action="create",
            manifest=w_inner,
            success_condition="status.phase == Succeeded",
            failure_condition="status.phase in (Failed, Error)",
        )
        with Steps(name="main"):
            steps = [Step(name=f"sub-workflow-1-{i}", template=w1_resource) for i in range(1)]

    # w_outer.create()
    w_inner.to_file("generated-workflows")
    w_outer.to_file("generated-workflows")

[flaviuvadan]

@rupello I tried the following and it works fine but I am confused about why it needs an input parameter. I think Argo might try to resolve the inner workflow inputs.parameters.message which is on the container only and get confused that it's actually specified by the DAG tasks rather than the outer / main workflow?

with Workflow(
    generate_name="inner-workflow-",
    entrypoint="diamond",
) as w_inner:
    echo = Container(
        name="echo",
        image="alpine:3.7",
        command=["echo", "{{inputs.parameters.message}}"],
        inputs=[Parameter(name="message")],
    )
    with DAG(name="diamond"):
        A = echo(name="A", arguments={"message": "A"})
        B = echo(name="B", arguments={"message": "B"})
        C = echo(name="C", arguments={"message": "C"})
        D = echo(name="D", arguments={"message": "D"})
        A >> [B, C] >> D

with Workflow(generate_name="outer-workflow-", entrypoint="main") as w_outer:
    w1_resource = Resource(
        name="w1-resource",
        action="create",
        manifest=w_inner,
        inputs=[Parameter(name="message", value="A")],  # <<<<<<<<<<<<< placeholder with default param value
        success_condition="status.phase == Succeeded",
        failure_condition="status.phase in (Failed, Error)",
    )
    with Steps(name="main"):
        w1_resource(name=f"sub-workflow-1-0")
        w1_resource(name=f"sub-workflow-1-1")

w_outer.create()

[greenbagels]

Hey, just checking in on this discussion; are you sure this example works fine when you run it? When I try it, every one of the inner echo containers echoes "A" (the default value). This mimics an issue that my org is having as well.

The only modifications I made to the example are connection-related settings for host, namespace, tokens, etc.

[greenbagels]

I guess another question is, if you encounter this issue, does it mean Argo resolves the "placeholder" parameters first, or is it a bug/unintended feature of Hera?

[flaviuvadan]

Apologies for the delayed reply, it turns out this is an Argo Workflows feature that's missing :( See discussion in the AW repo + suggestion of workaround: argoproj/argo-workflows#12634