Skip to content

Separate nightly CI from posted CI badge #2237

@ToucheSir

Description

@ToucheSir

Inspired from a discussion on Slack about how other packages handle the same. Flux frequently has to deal with this because new nightly releases often break Zygote, but Flux itself is rarely affected. There seem to be two main approaches to accomplishing this:

  1. Allow the nightly CI job to fail, e.g. Allow failures on GitHub Actions JuliaSmoothOptimizers/SolverTest.jl#6. This is simple to implement but does risk missing real failures if one doesn't dig beyond the big green checkmark.
  2. Separate nightly CI into its own job and show the non-nightly results badge, e.g. https://github.com/JuliaManifolds/Manifolds.jl/blob/master/.github/workflows/nightly.yml. This is more work but will still fail loudly for PRs.

A related question is whether we want to do the same for downstream tests. There are some known downstreams which are quite flaky, but since we don't include any downstream results in the README I'd assume this is less of a problem.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions