Skip to content

attempt to use local actions #10503

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
Actions
---

We use a number of local composite actions in `validate.yml`. Two of them simplify
`validate.yml` considerably; the other two less so, but will be useful in a future
tier-2 job to ensure that the main and tier 2 validate operations stay in sync.

We also use a central configuration file `.github/config.yml` containing the
various magic numbers/lists that bootstrap and validate use. The `Makefile`
also gets some information from it so it can stay in sync with `bootstrap.yml`.

The actions, found in `.github/actions` per convention, are:

- `cabal-setup`: does all the preliminary setup for a Cabal job
- `validate-build`: does `cabal-setup` and then builds cabal for validate
- `validate-old`: does `cabal-setup` and then validates a `validate-build` cabal against older GHCs
- `dogfooding`: does `cabal-setup` and then uses a `validate-build` cabal to build itself

As yet, there are no actions for the tests, because they're just a couple lines
of shell aside from `cabal-setup`. This may change in the future.

---

Workflows
---

The standard workflows are:

- `bootstrap.yml`: bootstrap a cabal from prepared JSONs (see `make bootstrap-jsons`)
- `validate.yml`: build a cabal with extra assertions and run the full test suite on it
- `changelogs.yml`: validate `changelog.d` files using [`changelog-d`]
- `dependabot.yml`: check `dependabot` configuration (sadly, not automatic; we lifted this from Ubuntu's CI)
- `lint.yml`: run `hlint` on cabal sources
- `format.yml`: check source formatting using Fourmolu v0.12
- `quick-jobs.yml`: various (formerly) quick checks
- `typos.yml`: look for typos in documentation
- `users-guide.yml`: generate the users guide, creating an artifact
- `whitespace.yml`: check for extraneous whitespace in various files
- `check-sdist.yml`: make sure cabal can be built against the `Cabal` bootlib (see e.g. #10931, #9863)

The validate workflow performs a number of tests on tier-1 platforms:

- on current GHCs (see `GHC_FOR_VALIDATE` and `GHC_FOR_VALIDATE_ONLY` in `config.yml`) it runs through the full suite of tests (`lib-tests`, `lib-suite`, `cli-tests`, and `cli-suite`)
- on older GHCs (see `GHC_FOR_VALIDATE_OLD`) it only runs `lib-suite-extras`, which is a cut-down test suite ("extras" refers to the fact that `validate.yml` historically referred to the old GHCs as `extra-ghc`)
- it builds but doesn't validate (for some reason) a static `cabal` on Alpine with MUSL
- it dogfoods `cabal` by having it build itself

The bootstrap workflow verifies that cabal can be built from pregenerated JSONs, for use in bootstrapping cabal on a new platform (since cabal is self-hosted).

---

Support tiers
---

Currently we support the following platforms as Tier 1:

- MacOS on AArch64
- X86-64 (aka AMD64)
- Windows (10 and 11)

Tier 2 platforms are:

- FreeBSD (AMD64 only)
- Alpine/MUSL static build
- MacOS on Intel
- X86 (deprecated)

We do not currently test on tier 2 platforms, but support for that is coming.
164 changes: 164 additions & 0 deletions .github/actions/cabal-setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
#
# Set up a workflow for building Cabal
#
# The only required input is the ghc version to build with (`ghc`). The other inputs are
# generally for special purposes:
#
# - an additional ghc version for testing, used by the old-versions tests (`extra-ghc`)
# - the `cabal.project` variant to use (`validate`)
# - `allow-newer` and `constraints` lines, for manual jobs
# - whether to build a static executable (for Alpine) (`static`)
# - shell (override on Windows because the default is the wrong version) (`shell`)
# - whether to use the cache (`with_cache`)
#
# There is only one output: the path to the installed (main) ghc.
#
# This is automatically done by the `validate-build` action, which takes the same
# parameters. It should be done directly for jobs that test the built `cabal`.
#

name: Cabal setup
description: Set up a workflow for Cabal

inputs:
ghc:
description: ghc version to use
required: true
extra-ghc:
description: additional ghc for tests
required: false
default: ''
project:
description: which cabal.project to use
required: false
default: 'validate'
allow-newer:
description: allow-newer line
required: false
default: ''
constraints:
description: constraints line
required: false
default: ''
static:
description: whether to build statically
required: false
default: 'false'
shell:
description: shell to use
required: false
default: 'bash'
with_cache:
description: whether to instantiate cache
required: false
default: 'true'

outputs:
ghc-exe:
description: Path to ghc installed by setup-haskell
value: ${{ steps.setup-haskell.outputs.ghc-exe }}

runs:
using: composite
steps:
- name: Make sure ghc is specified
if: inputs.ghc == ''
shell: ${{ inputs.shell }}
run: exit 1

- name: Work around existence of XDG directories (haskell-actions/setup#62)
if: runner.os == 'macOS'
shell: ${{ inputs.shell }}
run: |
rm -rf ~/.config/cabal
rm -rf ~/.cache/cabal

- name: "WIN: Setup TMP environment variable"
if: runner.os == 'Windows'
shell: ${{ inputs.shell }}
run: |
echo "TMP=${{ runner.temp }}" >> "$GITHUB_ENV"

# See https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#hackage-revisions
- name: Add manually supplied allow-newer
if: inputs.allow-newer != ''
shell: ${{ inputs.shell }}
run: |
echo "allow-newer: ${{ inputs.allow-newer }}" >> cabal.${{ inputs.project }}.project

- name: Add manually supplied constraints
if: inputs.constraints != ''
shell: ${{ inputs.shell }}
run: |
echo "constraints: ${{ inputs.constraints }}" >> cabal.${{ inputs.project }}.project

- name: Enable statically linked executables
if: inputs.static == 'true'
shell: ${{ inputs.shell }}
run: |
echo 'executable-static: true' >> cabal.${{ inputs.project }}.project

# must happen before the main setup so the correct ghc is default
- name: Install extra ghc for tests
if: inputs.extra-ghc != ''
uses: haskell-actions/setup@v2
with:
ghc-version: ${{ inputs.extra-ghc }}
cabal-version: '3.12.1.0' # see https://github.com/haskell/cabal/pull/10251

- uses: haskell-actions/setup@v2
id: setup-haskell
with:
ghc-version: ${{ inputs.ghc }}
cabal-version: '3.12.1.0'
# do we use this?
ghcup-release-channel: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.9.yaml

# See the following link for a breakdown of the following step
# https://github.com/haskell/actions/issues/7#issuecomment-745697160
- uses: actions/cache@v4
if: inputs.with_cache != 'false'
with:
# cabal-validate uses a special build dir
path: |
${{ steps.setup-haskell.outputs.cabal-store }}
dist-*
key: ${{ runner.os }}-${{ inputs.ghc }}-${{ github.sha }}
restore-keys: ${{ runner.os }}-${{ inputs.ghc }}-

# Needed by cabal-testsuite/PackageTests/Configure/setup.test.hs
- name: "MAC: Install Autotools"
if: runner.os == 'macOS'
shell: ${{ inputs.shell }}
run: |
brew install automake

# Needed by cabal-testsuite/PackageTests/Configure/setup.test.hs
- name: "WIN: Install Autotools"
if: runner.os == 'Windows'
shell: ${{ inputs.shell }}
run: |
/usr/bin/pacman --noconfirm -S autotools

- name: Set validate inputs
shell: ${{ inputs.shell }}
run: |
FLAGS="$COMMON_FLAGS"
if [[ "${{ inputs.ghc }}" == "$GHC_FOR_SOLVER_BENCHMARKS" ]]; then
FLAGS="$FLAGS --solver-benchmarks"
fi
if [[ "${{ inputs.ghc }}" == "$GHC_FOR_COMPLETE_HACKAGE_TESTS" ]]; then
FLAGS="$FLAGS --complete-hackage-tests"
fi
echo "FLAGS=$FLAGS" >> "$GITHUB_ENV"

- name: Canonicalize architecture
shell: ${{ inputs.shell }}
run: |
case ${{ runner.arch }} in
X86) arch=i386 ;;
X64) arch=x86_64 ;;
ARM64) arch=aarch64 ;;
*) echo "Unsupported architecture, please fix cabal-setup/action.yml" 2>/dev/null; exit 1 ;;
esac
echo "CABAL_ARCH=$arch" >> "$GITHUB_ENV"
61 changes: 61 additions & 0 deletions .github/actions/dogfooding/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
#
# Verify that a `cabal` can build itself. Use the `validate-build` action to
# build the `cabal` being tested. This is separated out so it can be reused by
# a future tier-2 job.
#
# The only required input is the ghc version to build with (`ghc`). The other inputs are
# generally for special purposes:
#
# - `allow-newer` and `constraints` lines, for manual jobs
# - shell (override on Windows because the default is the wrong version) (`shell`)
# - whether to use the cache (`with_cache`)
#
# There is only one output: the path to the installed (main) ghc.
#

name: Dogfooding cabal-install on a ghc/platform
description: Run a cabal-install uncached validate from a previously built binary

inputs:
ghc:
description: ghc version to use
required: true
shell:
description: shell to use
required: false
default: 'bash'
allow-newer:
description: allow-newer line
required: false
constraints:
description: constraints line
required: false

runs:
using: composite
steps:
- uses: ./.github/actions/cabal-setup
with:
ghc: ${{ inputs.ghc }}
shell: ${{ inputs.shell }}
allow-newer: ${{ inputs.allow_newer }}
constraints: ${{ inputs.constraints }}
# We don't use cache to force a build with a fresh store dir and build dir
# This way we check cabal can build all its dependencies
with_cache: 'false'

- name: Download cabal executable from workflow artifacts
uses: actions/download-artifact@v4
with:
name: cabal-${{ runner.os }}-${{ env.CABAL_ARCH }}
path: cabal-head

- name: Untar the cabal executable
shell: ${{ inputs.shell }}
run: |
tar -xzf "./cabal-head/cabal-head-${{ runner.os }}-$CABAL_ARCH.tar.gz" -C cabal-head

- name: Build using cabal HEAD
shell: ${{ inputs.shell }}
run: |
sh validate.sh $COMMON_FLAGS --with-cabal ./cabal-head/cabal -s build
102 changes: 102 additions & 0 deletions .github/actions/validate-build/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#
# Build `cabal` for validation tests
#
# The only required input is the ghc version to build with (`ghc`). The other inputs are
# generally for special purposes:
#
# - `allow-newer` and `constraints` lines, for manual jobs
# - whether to build a static executable (for Alpine) (`static`)
# - shell (override on Windows because the default is the wrong version) (`shell`)
# - whether to use the cache (`with_cache`)
#

name: Validate build
description: Build for a full validate on a ghc version

inputs:
ghc:
description: ghc version to use
required: true
allow-newer:
description: allow-newer line
required: false
constraints:
description: constraints line
required: false
static:
description: whether to build statically
required: false
default: 'false'
shell:
description: shell to use
required: false
default: 'bash'
with_cache:
description: whether to instantiate cache
required: false
default: 'true'

runs:
using: composite
steps:
- uses: ./.github/actions/cabal-setup
id: cabal-setup
with:
shell: ${{ inputs.shell }}
ghc: ${{ inputs.ghc }}
allow-newer: ${{ inputs.allow-newer }}
constraints: ${{ inputs.constraints }}
static: ${{ inputs.static }}
with_cache: ${{ inputs.with_cache }}

# The tool is not essential to the rest of the test suite. If
# hackage-repo-tool is not present, any test that requires it will
# be skipped.
# We want to keep this in the loop but we don't want to fail if
# hackage-repo-tool breaks or fails to support a newer GHC version.
- name: Install hackage-repo-tool
continue-on-error: true
shell: ${{ inputs.shell }}
run: |
cabal install --ignore-project hackage-repo-tool

- name: Validate build
shell: ${{ inputs.shell }}
run: |
echo ::group::Build
sh validate.sh $FLAGS -s build

- name: Tar cabal head executable
if: inputs.ghc == env.GHC_FOR_RELEASE
shell: ${{ inputs.shell }}
run: |
echo ::group::Tar
CABAL_EXEC=$(cabal list-bin --builddir=dist-newstyle-validate-ghc-${{ inputs.ghc }} --project-file=cabal.validate.project cabal-install:exe:cabal)
# We have to tar the executable to preserve executable permissions
# see https://github.com/actions/upload-artifact/issues/38
if [[ "${{ runner.os }}" == "Windows" ]]; then
# `cabal list-bin` gives us a windows path but tar needs the posix one
CABAL_EXEC=$(cygpath "$CABAL_EXEC")
fi
if [[ "${{ runner.os }}" == "macOS" ]]; then
# Workaround to avoid bsdtar corrupting the executable
# such that executing it after untar throws `cannot execute binary file`
# see https://github.com/actions/virtual-environments/issues/2619#issuecomment-788397841
sudo /usr/sbin/purge
fi
DIR=$(dirname "$CABAL_EXEC")
FILE=$(basename "$CABAL_EXEC")
CABAL_EXEC_TAR="cabal-head-${{ runner.os }}${{ inputs.static == 'true' && '-static' || '' }}-$CABAL_ARCH.tar.gz"
tar -czvf "$CABAL_EXEC_TAR" -C "$DIR" "$FILE"
echo "CABAL_EXEC_TAR=$CABAL_EXEC_TAR" >> "$GITHUB_ENV"

# We upload the cabal executable built with the ghc used in the release so we can:
# - Reuse it in the dogfooding job (although we could use the cached build dir)
# - Make it available in the workflow to make easier testing it locally
# (Using the cache in dogfooding would defeat its purpose, though.)
- name: Upload cabal-install executable to workflow artifacts
if: inputs.ghc == env.GHC_FOR_RELEASE
uses: actions/upload-artifact@v4
with:
name: cabal-${{ runner.os }}${{ inputs.static == 'true' && '-static' || '' }}-${{ env.CABAL_ARCH }}
path: ${{ env.CABAL_EXEC_TAR }}
Loading
Loading