Skip to content

Commit a317225

Browse files
geekosaurgeekosaur
committed
attempt to use local actions
Extremely experimental, and with all the limitations and restrictions I keep finding in GitHub Actions it'll probably fail in the messiest way it can. At present this is incomplete but sufficient to see if this has any chance of working to begin with. If it somehow does, I'll look into abstracting out the other sub-jobs, then making an overnight validate for Tier 2 platforms and probably a prerelease job (which would fix the recently revealed problem where if there is no need to rebase on merge, no prerelease is made).
1 parent cfbe545 commit a317225

File tree

9 files changed

+789
-377
lines changed

9 files changed

+789
-377
lines changed

‎.github/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
Actions
2+
---
3+
4+
We use a number of local composite actions in `validate.yml`. Two of them simplify
5+
`validate.yml` considerably; the other two less so, but will be useful in a future
6+
tier-2 job to ensure that the main and tier 2 validate operations stay in sync.
7+
8+
We also use a central configuration file `.github/config.yml` containing the
9+
various magic numbers/lists that bootstrap and validate use. The `Makefile`
10+
also gets some information from it so it can stay in sync with `bootstrap.yml`.
11+
12+
The actions, found in `.github/actions` per convention, are:
13+
14+
- `cabal-setup`: does all the preliminary setup for a Cabal job
15+
- `validate-build`: does `cabal-setup` and then builds cabal for validate
16+
- `validate-old`: does `cabal-setup` and then validates a `validate-build` cabal against older GHCs
17+
- `dogfooding`: does `cabal-setup` and then uses a `validate-build` cabal to build itself
18+
19+
As yet, there are no actions for the tests, because they're just a couple lines
20+
of shell aside from `cabal-setup`. This may change in the future.
21+
22+
---
23+
24+
Workflows
25+
---
26+
27+
The standard workflows are:
28+
29+
- `bootstrap.yml`: bootstrap a cabal from prepared JSONs (see `make bootstrap-jsons`)
30+
- `validate.yml`: build a cabal with extra assertions and run the full test suite on it
31+
- `changelogs.yml`: validate `changelog.d` files using [`changelog-d`]
32+
- `dependabot.yml`: check `dependabot` configuration (sadly, not automatic; we lifted this from Ubuntu's CI)
33+
- `lint.yml`: run `hlint` on cabal sources
34+
- `format.yml`: check source formatting using Fourmolu v0.12
35+
- `quick-jobs.yml`: various (formerly) quick checks
36+
- `typos.yml`: look for typos in documentation
37+
- `users-guide.yml`: generate the users guide, creating an artifact
38+
- `whitespace.yml`: check for extraneous whitespace in various files
39+
- `check-sdist.yml`: make sure cabal can be built against the `Cabal` bootlib (see e.g. #10931, #9863)
40+
41+
The validate workflow performs a number of tests on tier-1 platforms:
42+
43+
- on current GHCs (see `GHC_FOR_VALIDATE` and `GHC_FOR_VALIDATE_ONLY` in `config.yml`) it runs through the full suite of tests (`lib-tests`, `lib-suite`, `cli-tests`, and `cli-suite`)
44+
- on older GHCs (see `GHC_FOR_VALIDATE_OLD`) it only runs `lib-suite-extras`, which is a cut-down test suite ("extras" refers to the fact that `validate.yml` historically referred to the old GHCs as `extra-ghc`)
45+
- it builds but doesn't validate (for some reason) a static `cabal` on Alpine with MUSL
46+
- it dogfoods `cabal` by having it build itself
47+
48+
The bootstrap workflow verifies that cabal can be built from pregenerated JSONs, for use in bootstrapping cabal on a new platform (since cabal is self-hosted).
49+
50+
---
51+
52+
Support tiers
53+
---
54+
55+
Currently we support the following platforms as Tier 1:
56+
57+
- MacOS on AArch64
58+
- X86-64 (aka AMD64)
59+
- Windows (10 and 11)
60+
61+
Tier 2 platforms are:
62+
63+
- FreeBSD (AMD64 only)
64+
- Alpine/MUSL static build
65+
- MacOS on Intel
66+
- X86 (deprecated)
67+
68+
We do not currently test on tier 2 platforms, but support for that is coming.

‎.github/actions/cabal-setup/action.yml

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
#
2+
# Set up a workflow for building Cabal
3+
#
4+
# The only required input is the ghc version to build with (`ghc`). The other inputs are
5+
# generally for special purposes:
6+
#
7+
# - an additional ghc version for testing, used by the old-versions tests (`extra-ghc`)
8+
# - the `cabal.project` variant to use (`validate`)
9+
# - `allow-newer` and `constraints` lines, for manual jobs
10+
# - whether to build a static executable (for Alpine) (`static`)
11+
# - shell (override on Windows because the default is the wrong version) (`shell`)
12+
# - whether to use the cache (`with_cache`)
13+
#
14+
# There is only one output: the path to the installed (main) ghc.
15+
#
16+
# This is automatically done by the `validate-build` action, which takes the same
17+
# parameters. It should be done directly for jobs that test the built `cabal`.
18+
#
19+
20+
name: Cabal setup
21+
description: Set up a workflow for Cabal
22+
23+
inputs:
24+
ghc:
25+
description: ghc version to use
26+
required: true
27+
extra-ghc:
28+
description: additional ghc for tests
29+
required: false
30+
default: ''
31+
project:
32+
description: which cabal.project to use
33+
required: false
34+
default: 'validate'
35+
allow-newer:
36+
description: allow-newer line
37+
required: false
38+
default: ''
39+
constraints:
40+
description: constraints line
41+
required: false
42+
default: ''
43+
static:
44+
description: whether to build statically
45+
required: false
46+
default: 'false'
47+
shell:
48+
description: shell to use
49+
required: false
50+
default: 'bash'
51+
with_cache:
52+
description: whether to instantiate cache
53+
required: false
54+
default: 'true'
55+
56+
outputs:
57+
ghc-exe:
58+
description: Path to ghc installed by setup-haskell
59+
value: ${{ steps.setup-haskell.outputs.ghc-exe }}
60+
61+
runs:
62+
using: composite
63+
steps:
64+
- name: Make sure ghc is specified
65+
if: inputs.ghc == ''
66+
shell: ${{ inputs.shell }}
67+
run: exit 1
68+
69+
- name: Work around existence of XDG directories (haskell-actions/setup#62)
70+
if: runner.os == 'macOS'
71+
shell: ${{ inputs.shell }}
72+
run: |
73+
rm -rf ~/.config/cabal
74+
rm -rf ~/.cache/cabal
75+
76+
- name: "WIN: Setup TMP environment variable"
77+
if: runner.os == 'Windows'
78+
shell: ${{ inputs.shell }}
79+
run: |
80+
echo "TMP=${{ runner.temp }}" >> "$GITHUB_ENV"
81+
82+
# See https://github.com/haskell/cabal/blob/master/CONTRIBUTING.md#hackage-revisions
83+
- name: Add manually supplied allow-newer
84+
if: inputs.allow-newer != ''
85+
shell: ${{ inputs.shell }}
86+
run: |
87+
echo "allow-newer: ${{ inputs.allow-newer }}" >> cabal.${{ inputs.project }}.project
88+
89+
- name: Add manually supplied constraints
90+
if: inputs.constraints != ''
91+
shell: ${{ inputs.shell }}
92+
run: |
93+
echo "constraints: ${{ inputs.constraints }}" >> cabal.${{ inputs.project }}.project
94+
95+
- name: Enable statically linked executables
96+
if: inputs.static == 'true'
97+
shell: ${{ inputs.shell }}
98+
run: |
99+
echo 'executable-static: true' >> cabal.${{ inputs.project }}.project
100+
101+
# must happen before the main setup so the correct ghc is default
102+
- name: Install extra ghc for tests
103+
if: inputs.extra-ghc != ''
104+
uses: haskell-actions/setup@v2
105+
with:
106+
ghc-version: ${{ inputs.extra-ghc }}
107+
cabal-version: '3.12.1.0' # see https://github.com/haskell/cabal/pull/10251
108+
109+
- uses: haskell-actions/setup@v2
110+
id: setup-haskell
111+
with:
112+
ghc-version: ${{ inputs.ghc }}
113+
cabal-version: '3.12.1.0'
114+
# do we use this?
115+
ghcup-release-channel: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.8.yaml
116+
117+
# See the following link for a breakdown of the following step
118+
# https://github.com/haskell/actions/issues/7#issuecomment-745697160
119+
- uses: actions/cache@v4
120+
if: inputs.with_cache != 'false'
121+
with:
122+
# cabal-validate uses a special build dir
123+
path: |
124+
${{ steps.setup-haskell.outputs.cabal-store }}
125+
dist-*
126+
key: ${{ runner.os }}-${{ inputs.ghc }}-${{ github.sha }}
127+
restore-keys: ${{ runner.os }}-${{ inputs.ghc }}-
128+
129+
# Needed by cabal-testsuite/PackageTests/Configure/setup.test.hs
130+
- name: "MAC: Install Autotools"
131+
if: runner.os == 'macOS'
132+
shell: ${{ inputs.shell }}
133+
run: |
134+
brew install automake
135+
136+
# Needed by cabal-testsuite/PackageTests/Configure/setup.test.hs
137+
- name: "WIN: Install Autotools"
138+
if: runner.os == 'Windows'
139+
shell: ${{ inputs.shell }}
140+
run: |
141+
/usr/bin/pacman --noconfirm -S autotools
142+
143+
- name: Set validate inputs
144+
shell: ${{ inputs.shell }}
145+
run: |
146+
FLAGS="$COMMON_FLAGS"
147+
if [[ "${{ inputs.ghc }}" == "$GHC_FOR_SOLVER_BENCHMARKS" ]]; then
148+
FLAGS="$FLAGS --solver-benchmarks"
149+
fi
150+
if [[ "${{ inputs.ghc }}" == "$GHC_FOR_COMPLETE_HACKAGE_TESTS" ]]; then
151+
FLAGS="$FLAGS --complete-hackage-tests"
152+
fi
153+
echo "FLAGS=$FLAGS" >> "$GITHUB_ENV"
154+
155+
- name: Canonicalize architecture
156+
shell: ${{ inputs.shell }}
157+
run: |
158+
case ${{ runner.arch }} in
159+
X86) arch=i386 ;;
160+
X64) arch=x86_64 ;;
161+
ARM64) arch=aarch64 ;;
162+
*) echo "Unsupported architecture, please fix cabal-setup/action.yml" 2>/dev/null; exit 1 ;;
163+
esac
164+
echo "CABAL_ARCH=$arch" >> "$GITHUB_ENV"

‎.github/actions/dogfooding/action.yml

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
#
2+
# Verify that a `cabal` can build itself. Use the `validate-build` action to
3+
# build the `cabal` being tested. This is separated out so it can be reused by
4+
# a future tier-2 job.
5+
#
6+
# The only required input is the ghc version to build with (`ghc`). The other inputs are
7+
# generally for special purposes:
8+
#
9+
# - `allow-newer` and `constraints` lines, for manual jobs
10+
# - shell (override on Windows because the default is the wrong version) (`shell`)
11+
# - whether to use the cache (`with_cache`)
12+
#
13+
# There is only one output: the path to the installed (main) ghc.
14+
#
15+
16+
name: Dogfooding cabal-install on a ghc/platform
17+
description: Run a cabal-install uncached validate from a previously built binary
18+
19+
inputs:
20+
ghc:
21+
description: ghc version to use
22+
required: true
23+
shell:
24+
description: shell to use
25+
required: false
26+
default: 'bash'
27+
allow-newer:
28+
description: allow-newer line
29+
required: false
30+
constraints:
31+
description: constraints line
32+
required: false
33+
34+
runs:
35+
using: composite
36+
steps:
37+
- uses: ./.github/actions/cabal-setup
38+
with:
39+
ghc: ${{ inputs.ghc }}
40+
shell: ${{ inputs.shell }}
41+
allow-newer: ${{ inputs.allow_newer }}
42+
constraints: ${{ inputs.constraints }}
43+
# We don't use cache to force a build with a fresh store dir and build dir
44+
# This way we check cabal can build all its dependencies
45+
with_cache: 'false'
46+
47+
- name: Download cabal executable from workflow artifacts
48+
uses: actions/download-artifact@v4
49+
with:
50+
name: cabal-${{ runner.os }}-${{ env.CABAL_ARCH }}
51+
path: cabal-head
52+
53+
- name: Untar the cabal executable
54+
shell: ${{ inputs.shell }}
55+
run: |
56+
tar -xzf "./cabal-head/cabal-head-${{ runner.os }}-$CABAL_ARCH.tar.gz" -C cabal-head
57+
58+
- name: Build using cabal HEAD
59+
shell: ${{ inputs.shell }}
60+
run: |
61+
sh validate.sh $COMMON_FLAGS --with-cabal ./cabal-head/cabal -s build

‎.github/actions/validate-build/action.yml

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
#
2+
# Build `cabal` for validation tests
3+
#
4+
# The only required input is the ghc version to build with (`ghc`). The other inputs are
5+
# generally for special purposes:
6+
#
7+
# - `allow-newer` and `constraints` lines, for manual jobs
8+
# - whether to build a static executable (for Alpine) (`static`)
9+
# - shell (override on Windows because the default is the wrong version) (`shell`)
10+
# - whether to use the cache (`with_cache`)
11+
#
12+
13+
name: Validate build
14+
description: Build for a full validate on a ghc version
15+
16+
inputs:
17+
ghc:
18+
description: ghc version to use
19+
required: true
20+
allow-newer:
21+
description: allow-newer line
22+
required: false
23+
constraints:
24+
description: constraints line
25+
required: false
26+
static:
27+
description: whether to build statically
28+
required: false
29+
default: 'false'
30+
shell:
31+
description: shell to use
32+
required: false
33+
default: 'bash'
34+
with_cache:
35+
description: whether to instantiate cache
36+
required: false
37+
default: 'true'
38+
39+
runs:
40+
using: composite
41+
steps:
42+
- uses: ./.github/actions/cabal-setup
43+
id: cabal-setup
44+
with:
45+
shell: ${{ inputs.shell }}
46+
ghc: ${{ inputs.ghc }}
47+
allow-newer: ${{ inputs.allow-newer }}
48+
constraints: ${{ inputs.constraints }}
49+
static: ${{ inputs.static }}
50+
with_cache: ${{ inputs.with_cache }}
51+
52+
# The tool is not essential to the rest of the test suite. If
53+
# hackage-repo-tool is not present, any test that requires it will
54+
# be skipped.
55+
# We want to keep this in the loop but we don't want to fail if
56+
# hackage-repo-tool breaks or fails to support a newer GHC version.
57+
- name: Install hackage-repo-tool
58+
continue-on-error: true
59+
shell: ${{ inputs.shell }}
60+
run: |
61+
cabal install --ignore-project hackage-repo-tool
62+
63+
- name: Validate build
64+
shell: ${{ inputs.shell }}
65+
run: |
66+
echo ::group::Build
67+
sh validate.sh $FLAGS -s build
68+
69+
- name: Tar cabal head executable
70+
if: inputs.ghc == env.GHC_FOR_RELEASE
71+
shell: ${{ inputs.shell }}
72+
run: |
73+
echo ::group::Tar
74+
CABAL_EXEC=$(cabal list-bin --builddir=dist-newstyle-validate-ghc-${{ inputs.ghc }} --project-file=cabal.validate.project cabal-install:exe:cabal)
75+
# We have to tar the executable to preserve executable permissions
76+
# see https://github.com/actions/upload-artifact/issues/38
77+
if [[ "${{ runner.os }}" == "Windows" ]]; then
78+
# `cabal list-bin` gives us a windows path but tar needs the posix one
79+
CABAL_EXEC=$(cygpath "$CABAL_EXEC")
80+
fi
81+
if [[ "${{ runner.os }}" == "macOS" ]]; then
82+
# Workaround to avoid bsdtar corrupting the executable
83+
# such that executing it after untar throws `cannot execute binary file`
84+
# see https://github.com/actions/virtual-environments/issues/2619#issuecomment-788397841
85+
sudo /usr/sbin/purge
86+
fi
87+
DIR=$(dirname "$CABAL_EXEC")
88+
FILE=$(basename "$CABAL_EXEC")
89+
CABAL_EXEC_TAR="cabal-head-${{ runner.os }}${{ inputs.static == 'true' && '-static' || '' }}-$CABAL_ARCH.tar.gz"
90+
tar -czvf "$CABAL_EXEC_TAR" -C "$DIR" "$FILE"
91+
echo "CABAL_EXEC_TAR=$CABAL_EXEC_TAR" >> "$GITHUB_ENV"
92+
93+
# We upload the cabal executable built with the ghc used in the release so we can:
94+
# - Reuse it in the dogfooding job (although we could use the cached build dir)
95+
# - Make it available in the workflow to make easier testing it locally
96+
# (Using the cache in dogfooding would defeat its purpose, though.)
97+
- name: Upload cabal-install executable to workflow artifacts
98+
if: inputs.ghc == env.GHC_FOR_RELEASE
99+
uses: actions/upload-artifact@v4
100+
with:
101+
name: cabal-${{ runner.os }}${{ inputs.static == 'true' && '-static' || '' }}-${{ env.CABAL_ARCH }}
102+
path: ${{ env.CABAL_EXEC_TAR }}

0 commit comments

Comments
 (0)