Dockerized bazel builds with heavy use of bash and gnu tools

[veith4f]

Hello folks. I am still relatively new to bazel and just took over maintenance for a large build env that going forward I intend to fully dockerize in order to avoid having to consider different build platforms. Quite frankly, I think it is a bit of an anachronism to implement treatments for all the different platforms in todays heavily containerized world.

Many bazel library rules ship with their own binaries under the hood that implement simple stuff like writing yaml or json files and do so in a platform-aware way. Implementing such rules is very work intensive and there is a simpler, kind of obvious way to achieve determinism and reproducibility.

Bazel builds should be containerized, say in a debian container and bazel rules should make heavy use of bash scripting and gnu tooling. There should be a bash rule that allows running bash scripts and a large lib of rules that build on that bash rule to implement stuff like base64-encode/decode, yq, jq, sed, and all these other targets that sort of naturally suggest themselves.

I implemented the following bash rule and stamper rule that uses it to demonstrate how this approach can be leveraged to quickly build powerful bazel rules while at the same time ensuring a stability and reproducibility.

load("@bazel_skylib//lib:dicts.bzl", "dicts")

def _bash_impl(
        ctx,
        name=None,
        script=None,
        args=[],
        env={}):
    name = name or ctx.attr.name
    script = script or ctx.attr.script
    env = env or ctx.attr.env

    scriptfile = ctx.actions.declare_file("%s.sh" % ctx.label.name)
    ctx.actions.write(
        output = scriptfile,
        content = """
            (%s) > %s
        """ % (script, ctx.outputs.out.path)

    )

    ctx.actions.run(
        executable = "bash",
        outputs = [ctx.outputs.out],
        inputs = [scriptfile, ctx.info_file, ctx.version_file],
        arguments = [scriptfile.path],
        env = env
    )
    return [DefaultInfo(files = depset([ctx.outputs.out]))]


_bash_attrs = dicts.add({
    "script": attr.string(
        doc = "The script to run.",
        mandatory = True,
    ),
    "env": attr.string_dict(
        doc = "bash environment.",
        mandatory = False,
        allow_empty = True,
    ),
})

_bash_outputs = {
    "out": "%{name}-outputs.txt",
}

bash = rule(
    attrs = _bash_attrs,
    doc = ("This rule runs a script on passed parameters."),
    executable = False,
    outputs = _bash_outputs,
    implementation = _bash_impl,
)

load("//bazel/rules/bash:defs.bzl", "bash")

def stamp(name, inputs=[]):
    bash(
        name = name,
        script = """
        read -ra array <<< $INPUTS
        for STAMP in "${array[@]}"; do
            if [[ "$STAMP" =~ \\{([A-Z_]+)\\} ]]; then
                STAMPED=$(grep ${BASH_REMATCH[1]} bazel-out/stable-status.txt | cut -d" " -f 2)
                echo "${STAMP/\\{${BASH_REMATCH[1]}\\}/$STAMPED}"
            else
                echo $STAMP
            fi
        done
        """,
        env = {
            "INPUTS": " ".join(inputs)
        }
    )

What do you guys think? Isn't this the kind of preferred way to use bazel?