Skip to content

Split calico go-build image publish into parallel pipelines #636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jan 17, 2025
Merged

Split calico go-build image publish into parallel pipelines #636

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ca9d062
Split calico go-build image publish into parallel pipelines
hjiawei Jan 17, 2025
18cc4fc
Use f1-standard-2 for CI
hjiawei Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions .semaphore/promotions/calico-go-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: v1.0
name: Publish calico/go-build images
agent:
machine:
type: f1-standard-4
type: f1-standard-2
os_image: ubuntu2204

execution_time_limit:
Expand Down Expand Up @@ -37,10 +37,23 @@ blocks:
jobs:
- name: Linux multi-arch
commands:
- if [ -z "${SEMAPHORE_GIT_PR_NUMBER}" ]; then make -C images calico-go-build-cd CONFIRM=true; fi
- name: Trigger calico/go-build pin updates
- if [ -z "${SEMAPHORE_GIT_PR_NUMBER}" ]; then make -C images calico-go-build-cd VALIDARCHES=$ARCH CONFIRM=true; fi
matrix:
- env_var: ARCH
values: ["amd64", "arm64", "ppc64le", "s390x"]
- name: Publish calico/go-build multi-arch manifests
dependencies:
- Publish calico/go-build images
run:
when: "branch = 'master' OR tag =~ '^1\\.\\d+\\.\\d-llvm\\d+\\.\\d\\.\\d-k8s1\\.\\d+\\.\\d'"
task:
jobs:
- name: Linux multi-arch manifests
commands:
- if [ -z "${SEMAPHORE_GIT_PR_NUMBER}" ]; then make -C images push-calico-go-build-manifests CONFIRM=true; fi
- name: Trigger calico/go-build pin updates
dependencies:
- Publish calico/go-build multi-arch manifests
run:
when: "tag =~ '^1\\.\\d+\\.\\d-llvm\\d+\\.\\d\\.\\d-k8s1\\.\\d+\\.\\d'"
task:
Expand Down
4 changes: 0 additions & 4 deletions .semaphore/semaphore.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,6 @@ blocks:
- name: calico/go-build image
dependencies: []
task:
agent:
machine:
type: f1-standard-4
os_image: ubuntu2204
Comment on lines -52 to -55
Copy link
Collaborator Author

@hjiawei hjiawei Jan 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching back to f1-standard-2 runners only increases the CI time by ~4 minutes (f1-standard-2 completes one pipeline in ~16 minutes vs f1-standard-4 completes in ~12 minutes). We have 4x more f1-standard-2 runners so CI jobs are unlikely to starve.

env_vars:
# The branch to test the current go-build against
- name: CALICO_BRANCH
Expand Down
8 changes: 6 additions & 2 deletions images/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,13 @@ sub-calico-go-build-image-%:
$(MAKE) calico-go-build-image ARCH=$*

.PHONY: calico-go-build-cd
calico-go-build-cd: calico-go-build-image-all
calico-go-build-cd: calico-go-build-image
$(MAKE) BUILD_IMAGES=$(CALICO_GO_BUILD) cd-common

.PHONY: push-calico-go-build-manifests
push-calico-go-build-manifests: var-require-one-of-CONFIRM-DRYRUN var-require-all-BRANCH_NAME
$(MAKE) BUILD_IMAGES=$(CALICO_GO_BUILD) push-manifests IMAGETAG=$(if $(IMAGETAG_PREFIX),$(IMAGETAG_PREFIX)-)$(BRANCH_NAME) EXCLUDEARCH="$(EXCLUDEARCH)"

.PHONY: clean
clean:
rm -f $(QEMU_USER_STATIC_IMAGE_CREATED)
Expand All @@ -88,4 +92,4 @@ clean:

.PHONY: cd-common
cd-common: var-require-one-of-CONFIRM-DRYRUN var-require-all-BRANCH_NAME
$(MAKE) retag-build-images-with-registries push-images-to-registries push-manifests IMAGETAG=$(if $(IMAGETAG_PREFIX),$(IMAGETAG_PREFIX)-)$(BRANCH_NAME) EXCLUDEARCH="$(EXCLUDEARCH)"
$(MAKE) retag-build-images-with-registries push-images-to-registries IMAGETAG=$(if $(IMAGETAG_PREFIX),$(IMAGETAG_PREFIX)-)$(BRANCH_NAME) EXCLUDEARCH="$(EXCLUDEARCH)"
Loading