Add Scaleway packer provider

[Tomy2e]

Change description

This PR adds the Scaleway Packer Provider to the repo to allow users to provision CAPI images on Scaleway.

[k8s-ci-robot]

Welcome @Tomy2e!

It looks like this is your first PR to kubernetes-sigs/image-builder 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/image-builder has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @Tomy2e. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[AverageMarcus]

Hi @Tomy2e 👋
Thank you so much for this contribution. 💙

Before we approve I have a few things I'd like to ask:

1. Is there currently a CAPI provider for Scaleway? If so, is it officially supported by Scaleway?
2. You work for Scaleway, correct? Is this going to be officially supported by Scaleway or is this your personal contribution?
3. Would you (or any of your colleagues) be willing to be an official reviewer for this provider so you get pinged for any relevant PRs related to this provider?
4. Is there any chance of Scaleway donating infrastructure to the project to run tests against?

The reason I'm asking is we're (the maintainers of image-builder) are already spread very thin with the amount of providers and operating systems that need to be covered and our current test coverage isn't enough to provide confidence in all changes being safe. So before we bring in more providers it would be good to get an idea of what kind of support we'd be able to get to maintain this provider and what amount of work it might cost us.

If you prefer, we can discuss this on Slack or during our office hours sync.

(cc @mboersma @jsturtevant @drew-viles)

[Tomy2e]

Hi @AverageMarcus!

To answer your questions:

1. The CAPI provider for Scaleway was made public today and is available at: https://github.com/scaleway/cluster-api-provider-scaleway. Of course, it is officially supported and maintained by Scaleway.
2. I do work for Scaleway, and my contributions to the image-builder project will also be officially supported by Scaleway.
3. You can add me, @remyleone, and @Mia-Cross (members of the DevTools team at Scaleway) to the list of reviewers for the Scaleway provider.
4. I would love to and have asked internally to know if this is feasible.

I understand this is not easy to maintain that many providers, this is why we would like to help as much as we can 😄
@remyleone has started a thread on Slack to discuss in details about this subject.

[AverageMarcus]

As mentioned on Slack, I'm happy with this considering y'all willing to be added as reviewers for this provider (we can do this as another PR after this has merged).

Just one thing I wanted to confirm - Scaleway doesn't have support for Flatcar as an OS, right?

/ok-to-test
/lgtm
/assign @drew-viles @mboersma @jsturtevant

[AverageMarcus]

The pull-packer-validate is failing with the following:

Error: Failed to prepare build: "scaleway"
2 error(s) occurred:
* scaleway Project ID must be specified
* scaleway Secret Key must be specified

Any idea how we can avoid this when running packer validate ?

Edit: Ah! Looks like we need to add some "fake" values here:

image-builder/images/capi/scripts/ci-packer-validate.sh

Lines 34 to 41 in 7c6c12c

	AZURE_LOCATION=fake RESOURCE_GROUP_NAME=fake STORAGE_ACCOUNT_NAME=fake \
	DIGITALOCEAN_ACCESS_TOKEN=fake GCP_PROJECT_ID=fake \
	OCI_AVAILABILITY_DOMAIN=fake OCI_SUBNET_OCID=fake OCI_USER_FINGERPRINT=fake \
	OCI_TENANCY_OCID=fake OCI_USER_OCID=fake OCI_USER_KEY_FILE=/tmp/oci_api_key.pem \
	NUTANIX_ENDPOINT=fake NUTANIX_CLUSTER_NAME=fake NUTANIX_USERNAME=fake \
	NUTANIX_PASSWORD=fake NUTANIX_SUBNET_NAME=fake \
	HCLOUD_LOCATION=fake HCLOUD_TOKEN=fake \
	make validate-all

---

Please also run make json-sort to fix json-sort-check

[k8s-ci-robot]

New changes are detected. LGTM label has been removed.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from averagemarcus. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Tomy2e]

Thanks @AverageMarcus @Gnoale for your reviews!

I'm not sure why pull-gcp-all and pull-ova-all are still failing, is there something I can do to fix that?

@AverageMarcus Regarding your question about Flatcar support, I don't think there is something official on our side but it looks like we already have an image built for us here: https://www.flatcar.org/docs/latest/installing/community-platforms/scaleway/. I haven't personally tested it however. Do you think it's worth adding it to our image-builder integration? If so, we could probably do that in another PR.

[AverageMarcus]

I'm not sure why pull-gcp-all and pull-ova-all are still failing, is there something I can do to fix that?

Looks like the GCP one is failing because Ubuntu 20.04 is no longer available. @cpanato do you happen to know anything about this? Do we now need to drop 20.04 support?

googlecompute.ubuntu-2004: Error getting source image for instance creation: Could not find image, ubuntu-2004-lts, in projects

For the OVA one, it looks like there might be a bug that we need to fix (we've just re-enabled these tests today)

rm: cannot remove '/home/prow/go/src/sigs.k8s.io/image-builder/images/capi/packer/ova/linux/rhel/http/9/ks.cfg': No such file or directory

cc @chrischdi any chance you could take a look at this? 🙏

Do you think it's worth adding it to our image-builder integration? If so, we could probably do that in another PR.

If we can support Flatcar that'd be awesome as would get this provider mostly in-line with others. We have no rule on which OSs to support but would be nice to have Flatcar as an option. I'm totally happy for this to come later though, no need to block this PR for it :)

[AverageMarcus]

I've opened a PR to hopefully fix the GCP failure: #1774

The OVA one I'm actually not sure about 😕 If I'm understanding it correctly it's actually failing here. But I don't understand how it could 🙈

/test pull-ova-all

The Azure ones are now failing because of:

�[0;32m    azure-arm.sig-azurelinux-3-gen2: fatal: [default]: FAILED! => {"changed": false, "dest": "/var/lib/kubelet/credential-provider/acr-credential-provider", "elapsed": 0, "msg": "Request failed", "response": "HTTP Error 404: Not Found", "status_code": 404, "url": "https://github.com/kubernetes-sigs/cloud-provider-azure/releases/latest/download/azure-acr-credential-provider-linux-amd64"}�[0m

It looks like https://github.com/kubernetes-sigs/cloud-provider-azure has stopped publishing the binaries on the GitHub releases 😞 Looks like we're going to need to reach out to them to see if this was a mistake or a deliberate change in the releases. (Edit: Reached out on Slack)

(Sorry for so many issues 😅 today isn't going too well so far.)

[mboersma]

/test pull-gcp-all
/override pull-azure-sigs
/override pull-azure-vhds

GCP should skip Ubuntu 20.04 now, and the Azure builds are broken due to kubernetes-sigs/cloud-provider-azure#9192

[k8s-ci-robot]

@mboersma: Overrode contexts on behalf of mboersma: pull-azure-sigs, pull-azure-vhds

In response to this:

/test pull-gcp-all
/override pull-azure-sigs
/override pull-azure-vhds

GCP should skip Ubuntu 20.04 now, and the Azure builds are broken due to kubernetes-sigs/cloud-provider-azure#9192

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.