chore: add Makefile and docs around our dev/test processes (xenial) (#633)

* chore(tests): decouple bats tests from nvm.sh

* chore(tests): add a Makefile

* chore(docs): add docs for our test and dev processes

* chore(docs): remove bullet point from linting section

Author: JGAntunes

CONTRIBUTING.md

@@ -9,6 +9,28 @@ please read the [code of conduct](CODE_OF_CONDUCT.md).
 2. Wait for the Jenkins build to finish
 3. Create a PR in the [buildbot](https://github.com/netlify/buildbot) to bump [the version](https://github.com/netlify/buildbot/blob/a247edab7ead955cc27bb70ecc9f081e68f1aea6/script/docker-build.sh#L17) of the `build-image`.
 
+## Development
+
+### Linting
+
+[ShellCheck](https://github.com/koalaman/shellcheck) usage is recommended, however it is not enforced.
+
+### Tests
+
+We have a set of automated tests in [./tests](./tests). These are [bats](https://github.com/bats-core/bats-core) tests that we use to make assertions not only on the correct functioning of our bash/shell scripts, but also of the software provided by our Docker image. Any fix or feature should be accompanied by a set of tests to validate that those changes work as expected. For an overview on how [bats works see here](https://bats-core.readthedocs.io/en/stable/).
+
+### Developing
+
+We provide a [Makefile](./Makefile) with a set of utility targets to help with development.
+
+Some examples:
+
+- `make test` build the test docker image and run the tests in [TAP fromat](http://testanything.org/).
+- `make test-local` creates a volume of the tests directory and the build scripts and run the tests inside the container. Useful when developing locally.
+- `make test-local FILTER=<regex>` provide a filter regex string to your test execution in order to select a specific set of tests.
+- `make run` build the base image and run a bash shell in a container based in it in interactive mode.
+- `make run-local` volume the build scripts and run a bash shell in a container based in the build image.
+
 ## License
 
 By contributing to Netlify's build-image, you agree that your contributions will be licensed

Dockerfile

@@ -517,6 +517,11 @@ ADD --chown=buildbot package.json /opt/buildhome/test-env/package.json
 RUN cd /opt/buildhome/test-env && . ~/.nvm/nvm.sh && npm i &&\
     ln -s /opt/build-bin/run-build-functions.sh /opt/buildhome/test-env/run-build-functions.sh &&\
     ln -s /opt/build-bin/build /opt/buildhome/test-env/run-build.sh
+
 ADD --chown=buildbot tests /opt/buildhome/test-env/tests
 WORKDIR /opt/buildhome/test-env
-CMD . ~/.nvm/nvm.sh && npm test
+
+# Set `bats` as entrypoint
+ENTRYPOINT ["node_modules/.bin/bats"]
+# Set the default flags for `bats`
+CMD ["--recursive", "--timing", "--tap", "tests"]

Makefile

@@ -0,0 +1,40 @@
+.PHONY: test test-local build-tests build-base run run-local
+
+image = build-image
+test-image = $(image)-test
+
+test: build-tests ## Run tests.
+	docker run --rm -it \
+    $(test-image)
+
+
+# TODO conditionally build the `$(test-image)` if it doesn't exist
+test-local: ## Volume the tests directory and build scripts, and run the tests. Useful while developing locally
+	docker run --rm -it \
+    -v $(PWD)/tests:/opt/buildhome/test-env/tests:ro \
+    -v $(PWD)/run-build.sh:/opt/build-bin/build:ro \
+    -v $(PWD)/run-build-functions.sh:/opt/build-bin/run-build-functions.sh:ro \
+    $(test-image) \
+    --filter "$(FILTER)" --recursive --timing --pretty tests
+
+build-tests: ## Build test image, which includes all the test dependencies and tooling
+	docker build \
+		-t $(test-image) \
+		.
+
+build-base: ## Build base build-image docker image, without test dependencies and tooling
+	docker build \
+		-t $(image) \
+    --target build-image \
+		.
+
+run: build-base ## Run a bash shell in the build-image
+	docker run --rm -it \
+    $(image)
+
+# TODO conditionally build `$(image)` if it doesn't exist
+run-local: ## Volume the build scripts and run a bash shell in the build-image
+	docker run --rm -it \
+    -v $(PWD)/run-build.sh:/opt/build-bin/build:ro \
+    -v $(PWD)/run-build-functions.sh:/opt/build-bin/run-build-functions.sh:ro \
+    $(image)

README.md

@@ -6,7 +6,7 @@ If you're having problems with your build, you can also use these tools to test
 
 ## Available images
 
-Netlify maintains multiple build images for testing new development as well as supporting legacy builds. Each image uses a different version of Ubuntu Linux, with a slightly different list of included language and software versions. 
+Netlify maintains multiple build images for testing new development as well as supporting legacy builds. Each image uses a different version of Ubuntu Linux, with a slightly different list of included language and software versions.
 
 The following images are currently available:
 
@@ -84,6 +84,10 @@ This will create a `tmp` directory that will have the repo that the buildbot clo
 T=tmp/tmp.XXXXX ./test-tools/test-build.sh path/to/site/repo 'your build command'
 ```
 
+## Development
+
+Visit the [development section in our CONTRIBUTING.md](CONTRIBUTING.md#development)
+
 ## Contributing
 
 Pull requests welcome, as long as they're not overly specific to a niche use-case.

tests/helpers.sh

@@ -24,3 +24,10 @@ set_fixture_as_repo() {
 setup_tmp_dir() {
   mktemp -d
 }
+
+# Sources nvm.sh, in order for node and npm binaries to be accessible
+source_nvm() {
+  # Disable shellcheck's source file check
+  # shellcheck source=/dev/null
+  source ~/.nvm/nvm.sh
+}

tests/node/base.bats

@@ -6,11 +6,12 @@ load '../../node_modules/bats-support/load'
 load '../../node_modules/bats-assert/load'
 
 
-NODE_VERSION=12.18.0
-
-#Note: These binaries are accessible because we source `~/.nvm/nvm.sh` before running the `bats` tests
+setup() {
+  source_nvm
+}
 
 @test 'node version ${NODE_VERSION} is installed and available at startup' {
+  NODE_VERSION=12.18.0
   run node --version
   assert_success
   assert_output --partial $NODE_VERSION

tests/node/yarn.bats

@@ -14,6 +14,8 @@ setup() {
 
   # Load functions
   load '../../run-build-functions.sh'
+
+  source_nvm
 }
 
 teardown() {