Skip to content

Conversation

@mallardduck
Copy link
Member

@mallardduck mallardduck commented Apr 25, 2025

Per title, this adds a new chart for testing this image standalone from other charts.

The benefit of this is that our QA teams can focus specifically on this image when we work on releases for it. This untangles us from potential complications related to shipping RC versions of this image into other projects.

So by isolating the RC testing and QA process of this project within itself - while replicating the actions other charts will make - we can have the flexibility of not being dependent on a specific chart for testing, while ensuring that the testing we do is an accurate reflection of the charts that will use the image.


Issues Being Addressed

Platform Mismatch of Image and Binary

This happened due to an upstream bug that I didn't catch sooner, I added a tool for verifying this but it's only useful after an image is pushed to a registry. CI needs to happen (and fail if needed) before a push happens so that's not good enough.


TODO:

  • Ensure we can CI test images before pushing,
    • Add basic (unit based, essentially) CI for initial tests,
  • Ensure future release will have a new kubectl-test chart provided,
    • Make sure the example tasks cause errors if they fail,
    • Configure helm-package style command for this repo,
  • Add foundational e2e framework for testing

Example of Issues:

Platform Mismatch of Image and Binary

This can be replicated with version v4.0.1 or earlier as this was fixed in v4.0.2. So to replicate an error (and prove the CI script works):

skopeo copy --all docker://docker.io/rancher/kuberlr-kubectl:v4.0.1 oci-archive:kuberlr-kubect-4.0.1.oci
OCI_ARCHIVE_PATH=./kuberlr-kubect-4.0.1.oci ./scripts/verify-local-multiarch-image

Upon running that verify command it will use the known broken OCI image and you will see the ARM arch image fail with:

🔧 Processing linux/arm64
➡️  Extracting layer: sha256:a3671d3a9fb7f6e5c2d7bab91de34a7d39b345ffd8b3ecd984a74684341da01c
➡️  Extracting layer: sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
❌ Mismatch: Expected 'ARM aarch64', got: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, Go BuildID=hFrWgCcKkPxCltri3kRE/vY1VwNoMZzVXcGYU8YxB/xsMCHAXcU9CL24z7GCiF/Om_z5VXQ3wvGoLbDmce8, with debug_info, not stripped
✅ Match: ./ci/files/linux-arm64/kubectl1.32.0 is ARM aarch64
✅ Match: ./ci/files/linux-arm64/kubectl1.30.8 is ARM aarch64
✅ Match: ./ci/files/linux-arm64/kubectl1.31.4 is ARM aarch64
⚠️ Incomplete or mismatched files in linux/arm64

@mallardduck mallardduck force-pushed the qa-testing-chart branch 7 times, most recently from 215fef2 to c179a94 Compare May 27, 2025 21:13
@mallardduck
Copy link
Member Author

Closing in favor of: #117

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant