Skip to content

18.04 CML image failing to install terraform #773

@risktoparkrageram

Description

@risktoparkrageram

We are running CML using AWS and from last week have been getting errors when spinning up the runners like this:

cloud-init[3069]: tpi:setup.sh: sudo mv docker-credential-gcr /usr/bin/docker-credential-gcr\n[   81.521044]
cloud-init[3069]: tpi:setup.sh: curl -fsSL https://apt.releases.hashicorp.com/gpg\n[   81.521111]
cloud-init[3069]: tpi:setup.sh: sudo apt-key add -\n[   81.556216]
cloud-init[3069]: Warning: apt-key output should not be parsed (stdout is not a terminal)\n[   81.642723]
cloud-init[3069]: OK\n[   81.647318]
cloud-init[3069]: tpi:setup.sh: lsb_release -cs\n[   81.673421]
cloud-init[3069]: tpi:setup.sh: sudo apt-add-repository deb [arch=amd64] https://apt.releases.hashicorp.com bionic main\n         Starting Update APT News...\n         Starting Update the local ESM caches...\n[   82.100306]
cloud-init[3069]: Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease\n[   82.100457]
cloud-init[3069]: Hit:2 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic InRelease\n[   82.100620]
cloud-init[3069]: Hit:3 http://ppa.launchpad.net/git-core/ppa/ubuntu bionic InRelease\n[   82.103283]
cloud-init[3069]: Hit:4 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic-updates InRelease\n[   82.105690]
cloud-init[3069]: Hit:5 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic-backports InRelease\n[   82.107193]
cloud-init[3069]: Hit:6 https://download.docker.com/linux/ubuntu bionic InRelease\n[\u001b[0;32m  OK  \u001b[0m]
Started Update APT News.\n[   82.368015]
cloud-init[3069]: Ign:7 https://apt.releases.hashicorp.com bionic InRelease\n[   82.627173]
cloud-init[3069]: Err:8 https://apt.releases.hashicorp.com bionic Release\n[   82.628441]
cloud-init[3069]:   404  Not Found [IP: some-ip]\n[\u001b[0;32m  OK  \u001b[0m]
Started Update the local ESM caches.\n[   85.152879]
cloud-init[3069]: Reading package lists...\n[   85.180259]
cloud-init[3069]: E: The repository 'https://apt.releases.hashicorp.com bionic Release' does not have a Release file.\n[   85.181761]
cloud-init[3069]: tpi:setup.sh: sudo apt update\n[   85.186698]
cloud-init[3069]: WARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n         Starting Update APT News...\n         Starting Update the local ESM caches...\n[   85.251238]
cloud-init[3069]: Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease\n[   85.251381]
cloud-init[3069]: Hit:2 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic InRelease\n[   85.251510]
cloud-init[3069]: Hit:3 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic-updates InRelease\n[   85.254358]
cloud-init[3069]: Hit:4 http://eu-west-2.ec2.archive.ubuntu.com/ubuntu bionic-backports InRelease\n[   85.259454]
cloud-init[3069]: Hit:5 https://download.docker.com/linux/ubuntu bionic InRelease\n[   85.261302]
cloud-init[3069]: Ign:6 https://apt.releases.hashicorp.com bionic InRelease\n[   85.264582]
cloud-init[3069]: Err:7 https://apt.releases.hashicorp.com bionic Release\n[   85.264645]
cloud-init[3069]:   404  Not Found [IP: some-ip]\n[   85.294728]

With the image being set as aws_build_ubuntu_image = "*ubuntu-*-18.04-amd64-server-*", then this line points us to a APT repo that is no longer there, causing CML to fail!

The symptoms are that the EC2 runner gets spun up, but then gets terminated quite soon after that, whilst the github action eventually times out:

{"level":"info","message":"Plan: 1 to add, 0 to change, 0 to destroy."}
{"level":"info","message":"iterative_cml_runner.runner: Creating..."}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [10s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [20s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [30s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [40s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [50s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m0s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m10s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m20s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m30s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m40s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [1m50s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m0s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m10s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m20s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m30s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m40s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [2m50s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m0s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m10s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m20s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m30s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m40s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [3m50s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [4m0s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [4m10s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [4m20s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [4m30s elapsed]"}
{"level":"info","message":"iterative_cml_runner.runner: Still creating... [4m40s elapsed]"}
Error: The operation was canceled.

Please let me know if there is any more context I can provide, I am hopeful that the solution could be solved by updating the ubuntu version so it points to an available hashicorp repo, or allowing arbitrary ubuntu versions? Alternatively I would be very grateful for any workaround ideas you might have!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions