Unable to cache Docker Compose images

[moshevayner]

Hey there!

I'm creating a generic GitHub Action to be used internally within our org, and which will allow us to spin up our Docker Compose services within a GHA workflow and utilize cache as much as possible, to speed up build times.

Here's what I have so far:

name: Docker Compose Bake with GHA Cache
description: Build Docker Compose services using Docker Bake and GitHub Actions cache backend

inputs:
  working-directory:
    description: 'Working directory containing the docker-compose file'
    required: false
    default: '.'

runs:
  using: "composite"
  steps:
    - name: Detect the Compose File
      id: detect-compose
      shell: bash
      working-directory: ${{ inputs.working-directory }}
      run: |
        file=$(find . -name *compose.y*ml -print -quit)
        if [[ -z "$file" ]]; then
          echo "No docker-compose file found"
          exit 1
        fi
        echo "Found docker-compose file: $file"
        echo "file=$file" >> $GITHUB_OUTPUT

    - name: Generate Bake Cache File from Compose Config
      shell: bash
      working-directory: ${{ inputs.working-directory }}
      run: |
        echo "Generating docker-compose-cache.json from normalized Compose config..."
        
        echo '{ "target": {' > docker-compose-cache.json

        first=1
        yq '.services | keys[]' ${{ steps.detect-compose.outputs.file }} | while read -r service; do
          image=$(yq ".services[\"$service\"].image" ${{ steps.detect-compose.outputs.file }})

          if [[ "$image" == "null" ]]; then
            echo "Skipping $service - no image defined"
            continue
          fi

          [[ $first -eq 0 ]] && echo ',' >> docker-compose-cache.json
          first=0

          cat <<EOF >> docker-compose-cache.json
          "$service": {
            "cache-from": ["type=gha,scope=compose-$service"],
            "cache-to": ["type=gha,scope=compose-$service,mode=max"],
            "output": ["type=docker"]
          }
        EOF
        done

        echo '} }' >> docker-compose-cache.json

        echo "Final docker-compose-cache.json:"
        cat docker-compose-cache.json

    - name: Build Compose Services with Docker Bake
      uses: docker/bake-action@v6
      with:
        source: .
        workdir: ${{ inputs.working-directory }}
        files: |
          ${{ steps.detect-compose.outputs.file }}
          docker-compose-cache.json

    - name: Spin Up Docker Compose
      shell: bash
      working-directory: ${{ inputs.working-directory }}
      run: |
        docker compose -f ${{ steps.detect-compose.outputs.file }} up -d

And here's an example output of a bake file generated from the Generate Bake Cache File... step:

{ "target": {
  "elasticsearch": {
    "cache-from": ["type=gha,scope=compose-elasticsearch"],
    "cache-to": ["type=gha,scope=compose-elasticsearch,mode=max"],
    "output": ["type=docker"]
  }
,
  "mongo": {
    "cache-from": ["type=gha,scope=compose-mongo"],
    "cache-to": ["type=gha,scope=compose-mongo,mode=max"],
    "output": ["type=docker"]
  }
,
  "zookeeper": {
    "cache-from": ["type=gha,scope=compose-zookeeper"],
    "cache-to": ["type=gha,scope=compose-zookeeper,mode=max"],
    "output": ["type=docker"]
  }
,
  "kafka": {
    "cache-from": ["type=gha,scope=compose-kafka"],
    "cache-to": ["type=gha,scope=compose-kafka,mode=max"],
    "output": ["type=docker"]
  }
,
  "postgresql": {
    "cache-from": ["type=gha,scope=compose-postgresql"],
    "cache-to": ["type=gha,scope=compose-postgresql,mode=max"],
    "output": ["type=docker"]
  }
} }

However, it doesn't seem like anything is getting cached from the images, and every time I run this action- all the images are getting pulled again.

So I wanted to make sure I'm using the caching option in the correct way? Am I missing something?

As a side note, I saw that it's also possible to pass-in set as an input to the docker/bake-action action and configure the caching there, so I tried to do something like the below, but that resulted with an error Error: cannot parse bake definitions: ERROR: could not find any target matching '*'

    - name: Build Compose Services with Docker Bake
      uses: docker/bake-action@v6
      with:
        source: .
        workdir: ${{ inputs.working-directory }}
        load: true
        set: |
          *.cache-from=type=gha,scope=compose-${{ steps.detect-compose.outputs.file }}
          *.cache-to=type=gha,scope=compose-${{ steps.detect-compose.outputs.file }},mode=max
        files: |
          ${{ steps.detect-compose.outputs.file }}

I'd love to get some pointers on that.
Thanks!! 🙏🏼