Add native build to CI and MUSL build #2490

Quinn-With-Two-Ns · 2025-04-23T23:31:39Z

Add native build to CI and MUSL build

cretz · 2025-04-25T19:04:38Z

.github/workflows/ci.yml

@@ -188,3 +188,9 @@ jobs:

    - name: Run copyright and code format checks
      run: ./gradlew --no-daemon checkLicenseMain checkLicenses spotlessCheck
+
+  build_native_images:


Is it by intention that we are building native images for every CI run now or is this leftover from testing? If it is by intention, I would usually be concerned with unnecessarily using storage for all of these images, though I see we only retain for one day, but I wonder if it's even worth that vs just a workflow call option to not upload the artifacts.

Yes, I do want to build as part of CI so we don't discover issue when trying to cut a release. I would be OK only doing the builds post merge or not including artifacts for CI

Works for me, especially if it's concurrent and doesn't take longer than the existing jobs (doesn't seem to though that poor macos-arm one in last CI run took a really long time). I do think we should skip uploading the artifact though, but not a big deal.

cretz · 2025-04-25T19:09:06Z

.github/workflows/build-native-image.yml

Are the test server artifacts on this PR's CI run much larger than what we've released in the past? For instance, Windows x64 on this PR's CI run is 62.4MB (exe inside about the same size), but the one at https://github.com/temporalio/sdk-java/releases is 18.4MB (exe inside about the same size). I wonder if it's due to Java upgrade or if we're inadvertently not trimming something anymore or what.

hm good callout, not sure let me check

OK I think I addressed the size by adjusting the optimization level and reverting the extra reflection stuff I was including by mistake

Once you double check the size I will remove the artifacts from the CI build

It's still slightly bigger. For instance, Linux amd64 in this CI is 22.5MB vs 18.8MB in the last release. I'm ok with that if you are.

Switching topics (sorry, just easy to keep the same thread), I kinda figured the musl-based binary would be bigger than the libc-based one since it should statically link musl. Can you confirm that, unlike glibc build, we statically link musl? (meaning it could technically run on any Linux not just musl-based distros)

musl is statically linked

And I assume you're ok with the ~12% size-after-extracted increase? Checked extracted sizes, for Linux amd64 on release it's 60.3MB but here on CI it's 67.8MB. I don't think I have a big problem with this, though is a bit strange I think to increase sizes for every binary just to add a new musl binary (though I assume/understand it's mostly due to our switching of how we build w/ graal).

.github/workflows/build-native-image.yml

+    name: Build native test server
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - runner: ubuntu-latest
+            os_family: linux
+            arch: amd64
+            musl: true
+          - runner: ubuntu-latest
+            os_family: linux
+            arch: amd64
+            musl: false
+          - runner: macos-13
+            os_family: macOS
+            arch: amd64
+          - runner: macos-latest
+            os_family: macOS
+            arch: arm64
+          - runner: ubuntu-24.04-arm
+            os_family: linux
+            arch: arm64
+          - runner: windows-latest
+            os_family: windows
+            arch: amd64
+    runs-on: ${{ matrix.runner }}
+    steps:
+      - name: Checkout repo
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+          submodules: recursive
+          ref: ${{ env.INPUT_REF }}
+
+      - name: Set up Java
+        if: matrix.os_family != 'Linux'
+        uses: actions/setup-java@v4
+        with:
+          java-version: |
+            21
+            23
+          distribution: "graalvm"
+
+      - name: Set up Gradle
+        if: matrix.os_family != 'Linux'
+        uses: gradle/actions/setup-gradle@v4
+
+      - name: Build native test server (non-Docker)
+        if: matrix.os_family != 'Linux'
+        run: |
+          ./gradlew -PnativeBuild :temporal-test-server:nativeCompile
+
+      - name: Build native test server (Docker non-musl)
+        if: matrix.os_family == 'Linux' && matrix.musl == false
+        run: |
+          docker run \
+            --rm -w /github/workspace -v "$(pwd):/github/workspace" \
+            $(docker build -q ./docker/native-image) \
+            sh -c "./gradlew -PnativeBuild :temporal-test-server:nativeCompile"
+
+      - name: Build native test server (Docker musl)
+        if: matrix.os_family == 'Linux' && matrix.musl == true
+        run: |
+          docker run \
+            --rm -w /github/workspace -v "$(pwd):/github/workspace" \
+            $(docker build -q ./docker/native-image-musl) \
+            sh -c "./gradlew -PnativeBuild -PnativeBuildMusl  :temporal-test-server:nativeCompile"
+      # path ends in a wildcard because on windows the file ends in '.exe'
+      - name: Upload executable to workflow
+        uses: actions/upload-artifact@v4
+        with:
+          name: ${{ matrix.musl && format('{0}_{1}_musl', matrix.os_family, matrix.arch) || format('{0}_{1}', matrix.os_family, matrix.arch)}}
+          path: |
+            temporal-test-server/build/native/nativeCompile/temporal-test-server*
+          if-no-files-found: error
+          retention-days: 1


To address the issue, we will add a permissions block to the workflow file. This block will explicitly define the minimal permissions required for the workflow to function correctly. Based on the workflow's actions, it primarily interacts with repository contents (e.g., checking out code) and uploads artifacts. Therefore, the contents: read permission is sufficient for this workflow. We will add the permissions block at the root level of the workflow to apply it to all jobs.

.github/workflows/ci.yml

+    name: Build native test server
+    uses: ./.github/workflows/build-native-image.yml
+    with:
+      ref: ${{ github.event.pull_request.head.sha }}


To fix the issue, we will add a permissions block at the root of the workflow file to define the minimal permissions required for all jobs. Based on the workflow's actions, the contents: read permission is sufficient for most steps, as they involve checking out the repository and running tests. If any job requires additional permissions, we will define a specific permissions block for that job.

cretz

Nothing blocking, though the size increase for all the existing binaries is unfortunate.

Quinn-With-Two-Ns force-pushed the musl-build branch from 5c6c17b to 75b4c6e Compare April 25, 2025 15:15

Quinn-With-Two-Ns changed the title ~~Add native build to CI~~ Add native build to CI and MUSL build Apr 25, 2025

Quinn-With-Two-Ns marked this pull request as ready for review April 25, 2025 15:16

Quinn-With-Two-Ns requested a review from a team as a code owner April 25, 2025 15:16

cretz reviewed Apr 25, 2025

View reviewed changes

Quinn-With-Two-Ns force-pushed the musl-build branch from 90a7e9e to 23d79ed Compare May 28, 2025 23:39

github-advanced-security bot found potential problems May 28, 2025

View reviewed changes

Quinn-With-Two-Ns force-pushed the musl-build branch 2 times, most recently from 2375763 to 4ee467f Compare May 29, 2025 00:29

Quinn-With-Two-Ns requested a review from cretz June 3, 2025 22:10

cretz approved these changes Jun 4, 2025

View reviewed changes

Quinn-With-Two-Ns force-pushed the musl-build branch from ec940f9 to c189e8b Compare June 11, 2025 21:30

Add native build to CI

2fa7b45

Quinn-With-Two-Ns force-pushed the musl-build branch from c189e8b to 2fa7b45 Compare June 11, 2025 22:35

Quinn-With-Two-Ns added 2 commits June 11, 2025 15:36

fix protocloud

dccef90

enforce upload_artifact

3c2dc52

Quinn-With-Two-Ns merged commit bdc94f7 into master Jun 11, 2025
18 checks passed

Quinn-With-Two-Ns deleted the musl-build branch June 11, 2025 23:12

@@ -1,2 +1,4 @@
             name: Build native image
+            permissions:
+              contents: read
             defaults:

@@ -1,2 +1,4 @@
             name: Continuous Integration
+            permissions:
+              contents: read
             on:

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add native build to CI and MUSL build #2490

Add native build to CI and MUSL build #2490

Uh oh!

Quinn-With-Two-Ns commented Apr 23, 2025 •

edited

Loading

Uh oh!

cretz Apr 25, 2025

Uh oh!

Quinn-With-Two-Ns Apr 25, 2025

Uh oh!

cretz Apr 25, 2025

Uh oh!

cretz Apr 25, 2025 •

edited

Loading

Uh oh!

Quinn-With-Two-Ns Apr 25, 2025

Uh oh!

Quinn-With-Two-Ns Jun 3, 2025

Uh oh!

Quinn-With-Two-Ns Jun 3, 2025

Uh oh!

cretz Jun 4, 2025

Uh oh!

Quinn-With-Two-Ns Jun 4, 2025

Uh oh!

cretz Jun 4, 2025 •

edited

Loading

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

cretz left a comment

Uh oh!

Uh oh!

Uh oh!

Add native build to CI and MUSL build #2490

Add native build to CI and MUSL build #2490

Uh oh!

Conversation

Quinn-With-Two-Ns commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Apr 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

cretz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Quinn-With-Two-Ns commented Apr 23, 2025 •

edited

Loading

cretz Apr 25, 2025 •

edited

Loading

cretz Jun 4, 2025 •

edited

Loading