Infrastructure for Erlang/OTP GitLab-based CI/CD

[max-au]

Project description: https://docs.google.com/document/d/1uNgQw2fFxtYn7nqhuARKymWODOWGAZ4UQnmja7qkIK4

One approach to implement the CI/CD complimenting GitHub is to use GitLab in CI/CD mode: https://docs.gitlab.com/ee/user/project/integrations/github.html

Prototyping GitLab-based CI/CD for Erlang/OTP requires self-hosted instance of GitLab and a number of custom GitLab runners. The proposed solution is:

1. GitLab instance hosting: VM provided by DigitalOcean. Rationale: out of the box backup solution, inexpensive, good support. GitLab has Ultimate licensing program for non-profit organisations.
2. Test runners: bare metal servers running host-level virtualization solution (VMWare ESXi), with Linux, FreeBSD, Windows and more. Rationale: bound cost of bare-metal hosting.

Alternatives to GitLab considered:

1. GitHub Actions custom test runners attached to the Erlang/OTP repository. The alternative is still on the table, not moving forward for I lack the necessary experience with GHA. Volunteer help would be welcome to explore this more.
2. CirrusCI, SourceHut: same as above.

Alternatives to deploy test runners:

1. Autoscaling (e.g. via GitLab Runner bastion: https://www.digitalocean.com/community/tutorials/how-to-autoscale-gitlab-continuous-deployment-with-gitlab-runner-on-digitalocean ). This limits OS selection, and requires some mechanism to limit amount of compute time/money. Also many OTP test suites are written with multiple sleep calls, meaning that most of the time tests are sleeping, wasting compute time.
2. VM-hosted runners. This helps to avoid maintenance of bare-metal servers. But it also limits OS selection, and further limits what virtualization technologies can be utilised inside hosted VMs (e.g. nested virtualization capabilities are limited when renting a VM from cloud provider).

[max-au]

Infrastructure deployed so far:

• GitLab instance has been deployed to https://ci.erlef.org (DigitalOcean VM)
• FreeBSD test runner (DigitalOcean VM, custom runner with Jails)
• Ubuntu 20.04 test runner (custom VM, Docker runner)

Integration enabled:

• prototype integration between GitLab and a fork of Erlang/OTP repo has been set up: https://ci.erlef.org/erlang/otp Be aware this is not a mirror of the official erlang/otp (for security reasons)