How to use the image resulting from docker_builder in another tasks without an external registry?

[abravalheri]

Hello, I was wondering if anyone has some tips for the challenge I am currently facing. I tried to summarise it bellow:

I am trying to create a .cirrus.yml template that I can use across several projects.

One of the challenges that I have is to install packages on Windows and cache them, so the next time the CI runs, they don't have to be installed again. My previous attempts showed that simply using regular cache/fingerprint/etc makes the configuration very complex, so in order to avoid that I tried to use the dockerfile directive to build a container image that can be cached.

This is more or less a simplified version of my attempt:

# ...
windows_task:
  name: test (Windows - 3.9.10)
  depends_on: [windows_dockerfile]
  dockerfile_cache: {folder: ._CIRRUS_CI_FILES_3.9.10_}
  windows_container:
    dockerfile: ._CIRRUS_CI_FILES_3.9.10_/Dockerfile
    os_version: 2019
    # ...

# ...

windows_docker_script_task:  # Only used to generate a Dockerfile
  name: create Dockerfile for Windows (3.9.10)
  alias: windows_dockerfile
  container: {image: "python:3.10-bullseye"}
  dockerfile_cache:
    folder: ._CIRRUS_CI_FILES_3.9.10_
    populate_script: |
      mkdir ._CIRRUS_CI_FILES_3.9.10_
      cat >> ._CIRRUS_CI_FILES_3.9.10_/Dockerfile <<EOS
      FROM cirrusci/windowsservercore:2019
      RUN powershell -NoLogo -NoProfile -Command \
          choco install -y --no-progress python --version=3.9.10; \
          refreshenv; \
          $env:Path += ";c:\python39"; \
          $env:Path += ";c:\python39\Scripts";
          pip install --upgrade certifi; \
          pip install --upgrade pip tox pipx
    fingerprint_script: cat ._CIRRUS_CI_FILES_3.9.10_/Dockerfile

However this results in an error:

failed to retrieve ".CIRRUS_CI_FILES_3.9.10/Dockerfile": file does not exist

Since my intention here is that I can re-use this file in other projects, it should be standalone and to not depend on additional files (which includes a separated Dockerfile). Using an external registry is also something I would like to avoid for 2 reasons: a) there is a lot of hassle in using an external registry (subscription, secret management, etc) and b) being able to add tools to this container without having to re-build it and upload to a registry first is very handy.

I was hopping that by using a cache here I could share the file created in the auxiliary task to the point it could be used by the dockerfile directive... But apparently that is not possible.

Since Cirrus uses a Docker Builder task under the hood to process the dockerfile directive, I wonder if there is a way of achieving that using docker_builder. I had a look on the docs, but what is not clear to me is how to use the container that is being built in one task in another task without having to use an external registry (only relying on the internal cirrus cache)...

[fkorotkov]

I thinks Dockerfile as CI environment is what you are looking for. It should work with windows_containers as well but yeah, you can't generate the Dockerfiles on the fly. Cirrus parser uses GitHub API in order to fetch these Dockerfiles to calculate hash in order to figure out if we need to build and cache the images.