archiveLogs and artifactGC

[static-moonlight]

The logs in the artifact repository (when archiveLogs is enabled), doesn't seem to be removed by the artifact gc when the workflow doesn't use any artifacts. Is that intentional?

• Can I include logs when the artifact gc runs?
• Can I store the logs to a separate bucket?

[wesleyscholl]

@static-moonlight

Artifact GC and Archived Logs

Log artifacts not being removed by the Artifact Garbage Collection (GC) has been addressed in recent Argo Workflows releases.
In Argo Workflows version 3.4+, workflows can be configured to automatically delete artifacts, including archived logs. The Artifact Garbage Collection feature can be configured from the workflow and artifact levels.

Configuring Artifact GC for Logs

To include logs in the Artifact GC process, configure the artifactGC strategy in the workflow spec.

apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: artifact-gc-logs-
spec:
  entrypoint: main
  artifactGC:
    strategy: OnWorkflowCompletion  # or OnWorkflowDeletion
  templates:
    - name: main
      container:
        image: busybox
        command: [echo, "This is a log message"]
      archiveLocation:
        archiveLogs: true

The artifactGC strategy is set to OnWorkflowCompletion, meaning the logs (and other artifacts) will be deleted upon workflow completion.

Storing Logs in a Separate Bucket

To configure a separate artifact repository specifically for logs in the workflow-controller-configmap.

data:
  config: |
    artifactRepository:
      s3:
        bucket: your-logs-bucket
        endpoint: s3.amazonaws.com
        accessKeySecret:
          name: your-s3-secret
          key: accessKey
        secretKeySecret:
          name: your-s3-secret
          key: secretKey
        keyPrefix: logs/
    archiveLogs: true

This configuration defines a specific S3 bucket for logs.

Resources

• Argo Workflows Docs - Configuring Archive Logs
• Argo Workflows Docs - Artifact Garbage Collection
• Argo Workflows Docs - Artifacts
• artifactGC ignores archived logs when no artifacts are used #13784
• ArtifactGC not executing on archiveLogs when specified in Workflow level #13421
• Archive logs garbage collection #13338