How to let exit handler access stdout result and artifacts?

[static-moonlight]

This is a simplified version of what I'm trying to do. There might be a few yaml syntax errors in here, but that should not be an issue. Basically I have a dag workflow that creates multiple variants of raw input data. And when this is done, for each variant I want to send a message using an exit handler. To do that, I first need to build the message. And for that I need information and artifacts from the actual workflow. Even after looking into the documentation, I am pretty much confused how to do that. I marked the problematic spots in the exit handler with <??? [...] ???>

Note: The real thing is much more complex ;-). Even if it looks like this could be separated into 2 separate workflow executions, in reality the variants kind of depend on each other. And dependent on the raw data (and a few other factors), later steps (variants) can fail. But that should not prevent from uploading the parts that were completed and send a notification for them. Again: this is a simplified version ;-)

Can someone please enlighten me how to get the required data into the exit handler tasks? :) (especially when using dag and multiple dag tasks which use the same task template).

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
  [...]
spec:
  entrypoint: process-data
  onExit: notify
  templates:
    - name: process-data
      dag:
        tasks:
          - name: determine-id
            template: determine-id
          - name: download-raw-data
            template: download-data
            dependencies: [determine-id]
            arguments:
              parameters:
                - name: ID
                  value: "{{tasks.determine-id.outputs.result}}"
          - name: create-variant-a
            template: create-data
            dependencies: [determine-id]
            arguments:
              parameters:
                - name: ID
                  value: "{{tasks.determine-id.outputs.result}}"
                - name: VARIANT
                  value: A
              artifacts:
                - name: RAW_DATA
                  from: "{{tasks.download-raw-data.outputs.artifacts.RAW_DATA}}"
          - name: create-variant-b
            template: create-data
            dependencies: [determine-id]
            arguments:
              parameters:
                - name: ID
                  value: "{{tasks.determine-id.outputs.result}}"
                - name: VARIANT
                  value: B
              artifacts:
                - name: RAW_DATA
                  from: "{{tasks.download-raw-data.outputs.artifacts.RAW_DATA}}"
          - name: upload-variant-a
            template: upload-data
            dependencies: [create-variant-a]
            arguments:
              parameters:
                - name: ID
                  value: "{{tasks.determine-id.outputs.result}}"
                - name: VARIANT
                  value: A
              artifacts:
                - name: DATA
                  from: "{{tasks.create-variant-a.outputs.artifacts.DATA}}"
          - name: upload-variant-b
            template: upload-data
            dependencies: [create-variant-b]
            arguments:
              parameters:
                - name: ID
                  value: "{{tasks.determine-id.outputs.result}}"
                - name: VARIANT
                  value: B
              artifacts:
                - name: DATA
                  from: "{{tasks.create-variant-b.outputs.artifacts.DATA}}"

    - name: notify
      dag:
        tasks:
          - name: build-message-a
            template: build-message
            when: <??? how to determine if "upload-variant-a" dag task was successful ???>
            arguments:
              parameters:
                - name: ID
                  value: <??? how to get stdout result from "determine-id" dag task ???>
              artifacts:
                - name: METADATA
                  from: <??? how to get output artifact from "create-variant-a" dag task ???>
          - name: build-message-b
            template: build-message
            when: <??? how to determine if "upload-variant-b" dag task was successful ???>
            arguments:
              parameters:
                - name: ID
                  value: <??? how to get stdout result from "determine-id" dag task ???>
              artifacts:
                - name: METADATA
                  from: <??? how to get output artifact from "create-variant-b" dag task ???>
          - name: send-message-a
            template: send-message
            dependencies: [build-message-a]
            arguments:
              artifacts:
                - name: MESSAGE
                  from: "{{tasks.build-message-a.outputs.artifacts.MESSAGE}}"
          - name: send-message-b
            template: send-message
            dependencies: [build-message-b]
            arguments:
              artifacts:
                - name: MESSAGE
                  from: "{{tasks.build-message-b.outputs.artifacts.MESSAGE}}"

    # TASK TEMPLATES

    - name: determine-id
      container:
        image: id-manager
        command: [determine-id]

    - name: download-data
      inputs:
        parameters:
          - name: ID
      outputs:
        artifacts:
          - name: RAW_DATA
            path: /tmp/raw-data
      container:
        image: rclone
        command: [rclone,copy,"repository:raw-data/{{inputs.parameters.ID}}",/tmp/raw-data/]
        envFrom:
          - configMapRef:
              name: rclone
          - secretRef:
              name: rclone

    - name: create-data
      inputs:
        parameters:
          - name: ID
          - name: VARIANT
        artifacts:
          - name: RAW_DATA
            path: /tmp/raw-data
      outputs:
        artifacts:
          - name: DATA
            path: /tmp/data
          - name: METADATA
            path: /tmp/metadata
      container:
        image: data-procesor
        command: [create-data]
        env:
          - name: ID
            value: "{{inputs.parameters.ID}}"
          - name: VARIANT
            value: "{{inputs.parameters.VARIANT}}"

    - name: upload-data
      inputs:
        parameters:
          - name: ID
          - name: VARIANT
        artifacts:
          - name: DATA
            path: /tmp/data
      container:
        image: rclone
        command: [rclone,copy,/tmp/data/,"repository:data/{{inputs.parameters.ID}}/{{inputs.parameters.VARIANT}}"]
        envFrom:
          - configMapRef:
              name: rclone
          - secretRef:
              name: rclone

    - name: build-message
      inputs:
        parameters:
          - name: ID
        artifacts:
          - name: METADATA
            path: /tmp/metadata
      outputs:
        artifacts:
          - name: MESSAGE
            path: /tmp/message
      container:
        image: message-generator
        command: [generate-message]
        env:
          - name: ID
            value: "{{inputs.parameters.ID}}"
          - name: METADATA_FILE
            value: "/tmp/metadata"
          - name: MESSAGE_OUTPUT_FILE
            value: "/tmp/message"

    - name: send-message
      inputs:
        artifacts:
          - name: MESSAGE
            path: /tmp/metadata
      container:
        image: message-sender
        command: [send-message]
        envFrom:
          - configMapRef:
              name: messaging
          - secretRef:
              name: messaging

[static-moonlight]

In the end I didn't use the exit handler. I didn't get it to work and it was easier to just put those steps at the end of the DAG.