Not getting workflow.status in metrics

[danielpof]

Hi, I'm adding metrics to my workflows in the attempt to be able to monitor scheduled workflow failure, however using workflow.status in a metric is not showing the value I would expect.

Bellow is an example workflow:

metadata:
  name: fancy-dragon
  namespace: default
spec:
  metrics:
    prometheus:
    - name: workflow_duration
      help: "Workflow duration"
      labels:
      - key: workflow_template
        value: 'my-workflow-template'
      - key: workflow_namespace
        value: "default"
      gauge:
        realtime: true
        value: '{{workflow.duration}}'
    - name: result_status
      help: "Workflow result status"
      labels:
      - key: workflow_tempalte
        value: 'my-workflow-template'
      - key: workflow_namespace
        value: "default"
      - key: workflow_status
        value: '{{workflow.status}}'
      counter:
        value: "1"
  arguments:
    parameters:
      - name: message
        value: hello argo
  entrypoint: argosay
  templates:
    - name: argosay
      inputs:
        parameters:
          - name: message
            value: '{{workflow.parameters.message}}'
      metrics:
        prometheus:
        - name: task_duration
          help: "Task duration"
          labels:
          - key: workflow_template
            value: 'my-workflow-template'
          - key: workflow_namespace
            value: "default"
          - key: task_name
            value: "test-template"
          gauge:
            realtime: true
            value: '{{duration}}'
      container:
        name: main
        image: 'argoproj/argosay:v2'
        command:
          - /argosay
        args:
          - echo
          - '{{inputs.parameters.message}}'
  ttlStrategy:
    secondsAfterCompletion: 300
  podGC:
    strategy: OnPodCompletion

When running this workflow workflow_status label is missing, however in other workflows it has Running as it's value which is not one of the 3 states listed in the documentation.

Can workflows.status not be used in a metric label? Is there a better way to record success/failure per workflow type?

[lyauyeung]

I am experiencing the same issue! The custom metrics created only contain the workflow.status: Running when querying them in Grafana.

[anamarija97]

Getting the same problem. When new pipeline is run and is failed it is not even showing in Prometheus or it is showing me as Running for failed and running pipeline.

[agilgur5]

however in other workflows it has Running as it's value which is not one of the 3 states listed in the documentation.

The documentation you're referencing is for exit handlers, but you're not using an exit handler. Prior to exit, the Workflow is indeed Running