CI: Build more variants of the executable (static libc, macos arm, macos universal) #2768
Conversation
cprecioso
force-pushed
the
cprecioso/push-kqnlxsmqzrqq
branch
19 times, most recently
from
495aaf6 to
511b4b3
Compare
cprecioso
force-pushed
the
cprecioso/push-kqnlxsmqzrqq
branch
from
511b4b3 to
b048477
Compare
cprecioso
changed the title
cprecioso
force-pushed
the
cprecioso/push-kqnlxsmqzrqq
branch
from
18d1d7e to
5592b5f
Compare
| with: | ||
| cabal-project: dev-tool.project |
There was a problem hiding this comment.
Leftover from previous version of the action
There was a problem hiding this comment.
Left a bit of comments! From what I get this is only part of the big change, so there isn't that much to review actually.
No. The syscall interface is an internal API in macOS, so we must dynamically link to libSystem.dylib, which is the public API. Fortunately, libSystem is much lighter than libc, and quite stable between releases, so it is not a big problem there.
-> let's add this to the code somewhere, as a comment!
So we don't depend on GitHub Action runners anymore. They keep updating them, changing the available software, and deprecating old runners. If we want to keep building on an old Glibc, we need pin the environment through Docker.
-> this should also be explained in the comment in the code!
There was a problem hiding this comment.
@cprecioso looking great all together! This is going to be very nice. I like a lot the whole deal with this being a separate workflow, it is much simpler to comprehend, and uploading artifacts is sweet. I don't think there is so much dependencies being installed as you mentioned, it isn't that much really.
Let's work through the comments, but there is nothing major!
Oh and if I got it right, you plan to then delete this logic from wasp-ci.yaml and of course the artifacts generated by this one, but you are leaving that step for some next PR?
|
@Martinsos addressed comments
|
Deploying wasp-docs-on-main with
|
| Latest commit: |
a6cfa58
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://86fce437.wasp-docs-on-main.pages.dev |
| Branch Preview URL: | https://cprecioso-push-kqnlxsmqzrqq.wasp-docs-on-main.pages.dev |
@cprecioso have you handled this comment? |
|
@Martinsos ready
Yes! Sorry I forgot the rocket. |
No worries, I ask because it is much faster for me to see the rocket than actually go hunt it down in the code changes, especially for comments like this where I am confident you are going to do a good job so I don't need to see the result, just that it was done / not skipped. |
|
All taken care of, will merge this and #2773 when the CI is green |
3 participants
Description
Context: RFC
This PR extracts the building and packaging logic from
wasp-ci.yamlto its ownwaspc-build.yaml.It tries to simplify the steps as much as possible. The intention is that the
stepsare easily readable and understandable in a single screenful.Most complexity lays in how we define the environments we run in, as we declare there the OS-specific dependencies we need to install.
Through this abstraction, we can define in a single place all our build targets, keeping compatibility workarounds to a minimum:
wasp-linux-x86_64: built on a Ubuntu 20.04 Docker image so we have a lowglibcrequirementwasp-linux-x86_64-static: (NEW) built on an Alpine image and with the--enable-static-buildcabal flag, so musl gets embedded into the binary and we're completely independent of the user's distrowasp-darwin-x86_64: no changes, built on the GitHub macOS runnerwasp-darwin-amd64: (NEW) we use the new GitHub macOS ARM runners to build an Apple Silicon binarywasp-darwin-universal: (NEW) I do an extra step at the end of picking up both macOS binaries and creating a Universal binary out of both of them.All binaries are uploaded as artifacts of the workflow run, which means they are available to later steps. They are also accessible to us through the workflow's summary page.
Important
This workflow is not active yet. We can run it manually through GitHub's interface when it comes to the default branch. I first sent the approach so we can discuss it by itself. The automation is complete and available at #2773.
FAQs
Why do we need to manually specify so many dependencies when we weren't doing it before?
Before, we were building on the GitHub runners, which come with a lot of dependencies already installed (even GHC!). Now we're building inside Docker images, and they are usually bare-bones, so they are missing many basic dependencies (e.g. Ubuntu is missing
curland Alpine is missingbash).Why build inside of Docker?
So we don't depend on GitHub Action runners anymore. They keep updating them, changing the available software, and deprecating old runners. If we want to keep building on an old Glibc, we need pin the environment through Docker.
Why do we build an ARM image for macOS and not Linux?
GHC 8.10.7 has support for ARM on macOS but not on Linux. Updating to a newer version (#1446) will enable us to do ARM binaries for Linux.
Why are we now able to build executables for macOS on ARM?
New images from GitHub became available.
Why have two binaries for Linux and three for macOS?
These are implemented in order for us to continue building in different environments and testing before release. We don't need to publish all of them.
I'd suggest to publish the Linux static and the macOS universal only. If we agree on that we can remove the others.
Can we statically link the macOS binaries?
No. The syscall interface is an internal API in macOS, so we must dynamically link to
libSystem.dylib, which is the public API. Fortunately, libSystem is much lighter than libc, and quite stable between releases, so it is not a big problem there.Why extract this into a new workflow?
wasp-ciis already quite heavy to read through, and full of workarounds. Having the build there would have been more messy than helpful. We can still call to this build from other workflows, so we don't lose functionality.