Not able to build Quarkus Native image using container build while using dynatrace-native-maven-plugin

[viswatejan]

Describe the bug

1. Added dynatrace-native-maven-plugin to the pom.xml to enable Dynatrace observability on the native image as documented here.

   <profile>
       <id>dynatrace-native</id>
       <build>
           <plugins>
               <plugin>
                   <groupId>com.dynatrace.buildtools.graalnative</groupId>
                   <artifactId>dynatrace-native-maven-plugin</artifactId>
                   <version>2.0.1</version>
                   <executions>
                       <execution>
                           <goals>
                               <goal>setup-build-agent</goal>
                               <goal>copy-runtime-agent</goal>
                           </goals>
                           <configuration>
                               <agentZip>./agent.zip</agentZip>
                           </configuration>
                       </execution>
                   </executions>
                   <extensions>true</extensions>
               </plugin>
           </plugins>
       </build>
   </profile>

2. Ran the below script to use container build option to build Quarkus native image.

   mvn clean package -Pdynatrace-native -Dnative -Dquarkus.native.container-build=true -Dquarkus.native.builder-image=graalvm -DskipTests

3. Build failed with the following log

   [INFO] Scanning for projects...
   [INFO] [dynatrace-native-maven-plugin] Detected Quarkus plugin
   [INFO] 
   [INFO] -----------< com.example:quarkus-project >-----------
   [INFO] Building quarkus-project 0.0.1-SNAPSHOT
   [INFO]   from pom.xml
   [INFO] --------------------------------[ jar ]---------------------------------
   [INFO] 
   [INFO] --- clean:3.2.0:clean (default-clean) @ quarkus-project ---
   [INFO] 
   [INFO] --- jacoco:0.8.11:prepare-agent (default-prepare-agent) @ quarkus-project ---
   [INFO] argLine set to -javaagent:$HOME/.m2/repository/org/jacoco/org.jacoco.agent/0.8.11/org.jacoco.agent-0.8.11-runtime.jar=destfile=$HOME/Work/Projects/quarkus-project/target/jacoco.exec,exclclassloader=*QuarkusClassLoader
   [INFO] 
   [INFO] --- dynatrace-native:2.0.1:setup-build-agent (default) @ quarkus-project ---
   [INFO] [dynatrace-native-maven-plugin] Prepared Dynatrace agent directory: '$HOME/Work/Projects/quarkus-project/target/tmp'
   [INFO] 
   [INFO] --- resources:3.3.1:resources (default-resources) @ quarkus-project ---
   [INFO] Copying 1 resource from src/main/resources to target/classes
   [INFO] 
   [INFO] --- quarkus:3.16.3:generate-code (default) @ quarkus-project ---
   Downloading from snapshots: https://example.com:8081/artifactory/maven-repos/com/example/quarkus-project/0.0.1-SNAPSHOT/maven-metadata.xml
   [INFO] 
   [INFO] --- dynatrace-native:2.0.1:copy-runtime-agent (default) @ quarkus-project ---
   [INFO] 
   [INFO] --- compiler:3.12.1:compile (default-compile) @ quarkus-project ---
   [INFO] Recompiling the module because of changed source code.
   [INFO] Compiling 31 source files with javac [debug target 21] to target/classes
   [INFO] 
   [INFO] --- quarkus:3.16.3:generate-code-tests (default) @ quarkus-project ---
   [INFO] 
   [INFO] --- resources:3.3.1:testResources (default-testResources) @ quarkus-project ---
   [INFO] Copying 3 resources from src/test/resources to target/test-classes
   [INFO] 
   [INFO] --- compiler:3.12.1:testCompile (default-testCompile) @ quarkus-project ---
   [INFO] Recompiling the module because of changed dependency.
   [INFO] Compiling 11 source files with javac [debug target 21] to target/test-classes
   [INFO] 
   [INFO] --- surefire:3.2.2:test (default-test) @ quarkus-project ---
   [INFO] Tests are skipped.
   [INFO] 
   [INFO] --- jar:3.4.1:jar (default-jar) @ quarkus-project ---
   [INFO] Building jar: $HOME/Work/Projects/quarkus-project/target/quarkus-project.jar
   [INFO] 
   [INFO] --- quarkus:3.16.3:build (default) @ quarkus-project ---
   [INFO] [io.quarkus.deployment.pkg.steps.JarResultBuildStep] Building native image source jar: $HOME/Work/Projects/quarkus-project/target/quarkus-project-native-image-source-jar/quarkus-project-runner.jar
   [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Building native image from $HOME/Work/Projects/quarkus-project/target/quarkus-project-native-image-source-jar/quarkus-project-runner.jar
   [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Using docker to run the native image builder
   [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildContainerRunner] Pulling builder image 'quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:jdk-21'
   jdk-21: Pulling from quarkus/ubi-quarkus-graalvmce-builder-image
   0c03fc0a30d2: Already exists 
   09767875e5b0: Pull complete 
   8eeb531e8a5b: Pull complete 
   24a6bf6902d1: Pull complete 
   b1bf710de9a1: Pull complete 
   6992685f18b3: Pull complete 
   86397256a5bf: Pull complete 
   9cae68f268f8: Pull complete 
   22d6a80c7ea0: Pull complete 
   e52b1e43c04b: Pull complete 
   90753ae0df31: Pull complete 
   47304f93d65b: Pull complete 
   53e8abd0a0c8: Pull complete 
   4f4fb700ef54: Pull complete 
   Digest: sha256:73615baf56b4980a6215ab3aec008218c921ab24f7939798ce735fba9ebf87ab
   Status: Downloaded newer image for quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:jdk-21
   quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:jdk-21
   [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on GRAALVM 23.1 JDK 21.0.2+13-jvmci-23.1-b30
   [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildRunner] docker run --env LANG=C --rm --user 504:20 -v $HOME/Work/Projects/quarkus-project/target/quarkus-project-native-image-source-jar:/project:z --name build-native-yCvRY quay.io/quarkus/ubi-quarkus-graalvmce-builder-image:jdk-21 -J-Dsun.nio.ch.maxUpdateArraySize=100 -J-Djava.util.logging.manager=org.jboss.logmanager.LogManager -J-Dlogging.initial-configurator.min-level=500 -J-Dvertx.logger-delegate-factory-class-name=io.quarkus.vertx.core.runtime.VertxLogDelegateFactory -J-Dvertx.disableDnsResolver=true -J-Dio.netty.leakDetection.level=DISABLED -J-Dio.netty.allocator.maxOrder=3 -J-Duser.language=en -J-Duser.country=US -J-Dfile.encoding=UTF-8 --features=io.quarkus.runner.Feature,io.quarkus.runtime.graal.DisableLoggingFeature -J--add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED -J--add-exports=java.security.jgss/sun.security.jgss=ALL-UNNAMED -J--add-opens=java.base/java.text=ALL-UNNAMED -J--add-opens=java.base/java.io=ALL-UNNAMED -J--add-opens=java.base/java.lang.invoke=ALL-UNNAMED -J--add-opens=java.base/java.util=ALL-UNNAMED -H:+UnlockExperimentalVMOptions -H:BuildOutputJSONFile=quarkus-project-runner-build-output-stats.json -H:-UnlockExperimentalVMOptions -H:+UnlockExperimentalVMOptions -H:+GenerateBuildArtifactsFile -H:-UnlockExperimentalVMOptions --strict-image-heap -cp $HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/any/java/oneagent-graalnative-feature.jar --features=com.dynatrace.graalnative.features.OneAgentGraalNativeFeature --enable-url-protocols=http,https -J-agentpath:$HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so -H:+UnlockExperimentalVMOptions -H:+AllowFoldMethods -H:-UnlockExperimentalVMOptions -J-Djava.awt.headless=true --no-fallback --link-at-build-time -H:+UnlockExperimentalVMOptions -H:+ReportExceptionStackTraces -H:-UnlockExperimentalVMOptions -H:-AddAllCharsets --enable-url-protocols=http,https --enable-monitoring=heapdump -H:+UnlockExperimentalVMOptions -H:-UseServiceLoaderFeature -H:-UnlockExperimentalVMOptions -J--add-exports=org.graalvm.nativeimage/org.graalvm.nativeimage.impl=ALL-UNNAMED --exclude-config io\.netty\.netty-codec /META-INF/native-image/io\.netty/netty-codec/generated/handlers/reflect-config\.json --exclude-config io\.netty\.netty-handler /META-INF/native-image/io\.netty/netty-handler/generated/handlers/reflect-config\.json quarkus-project-runner -jar quarkus-project-runner.jar
   Error occurred during initialization of VM
   Could not find agent library $HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so in absolute path, with error: $HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so: cannot open shared object file: No such file or directory
   [INFO] ------------------------------------------------------------------------
   [INFO] BUILD FAILURE
   [INFO] ------------------------------------------------------------------------
   [INFO] Total time:  01:31 min
   [INFO] Finished at: 2024-12-31T10:04:46-06:00
   [INFO] ------------------------------------------------------------------------
   [ERROR] Failed to execute goal io.quarkus.platform:quarkus-maven-plugin:3.16.3:build (default) on project quarkus-project: Failed to build quarkus application: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
   [ERROR]         [error]: Build step io.quarkus.deployment.pkg.steps.NativeImageBuildStep#build threw an exception: io.quarkus.deployment.pkg.steps.NativeImageBuildStep$ImageGenerationFailureException: Image generation failed. Exit code: 1
   [ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.imageGenerationFailed(NativeImageBuildStep.java:487)
   [ERROR]         at io.quarkus.deployment.pkg.steps.NativeImageBuildStep.build(NativeImageBuildStep.java:278)
   [ERROR]         at java.base/java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:733)
   [ERROR]         at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:856)
   [ERROR]         at io.quarkus.builder.BuildContext.run(BuildContext.java:256)
   [ERROR]         at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
   [ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2675)
   [ERROR]         at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2654)
   [ERROR]         at org.jboss.threads.EnhancedQueueExecutor.runThreadBody(EnhancedQueueExecutor.java:1627)
   [ERROR]         at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1594)
   [ERROR]         at java.base/java.lang.Thread.run(Thread.java:1583)
   [ERROR]         at org.jboss.threads.JBossThread.run(JBossThread.java:499)
   [ERROR] -> [Help 1]
   [ERROR] 
   [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
   [ERROR] Re-run Maven using the -X switch to enable full debug logging.
   [ERROR] 
   [ERROR] For more information about the errors and possible solutions, please read the following articles:
   [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
   

Expected behavior

Should successfully build the native image.

It will be better if we have a configuration property to enable mounting the entire target folder.

Actual behavior

Build failed

Directory /target/tmp, where the dynatrace plugin copied its agent is not included in the container-build volume mount (only mounting -v $HOME/Work/Projects/quarkus-project/target/quarkus-project-native-image-source-jar:/project:z) and so it is failing to find the dynatrace agent files to successfully build the native image.

How to Reproduce?

Go to Quarkus Getting Started and create a sample app.

Follow the steps included in the description section to reproduce the issue.

Output of uname -a or ver

Darwin macx-*** 24.2.0 Darwin Kernel Version 24.2.0: Fri Dec 6 18:41:43 PST 2024; root:xnu-11215.61.5~2/RELEASE_X86_64 x86_64

Output of java -version

java version "21.0.5" 2024-10-15 LTS Java(TM) SE Runtime Environment Oracle GraalVM 21.0.5+9.1 (build 21.0.5+9-LTS-jvmci-23.1-b48) Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.5+9.1 (build 21.0.5+9-LTS-jvmci-23.1-b48, mixed mode, sharing)

Mandrel or GraalVM version (if different from Java)

java version "21.0.5" 2024-10-15 LTS Java(TM) SE Runtime Environment Oracle GraalVM 21.0.5+9.1 (build 21.0.5+9-LTS-jvmci-23.1-b48) Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 21.0.5+9.1 (build 21.0.5+9-LTS-jvmci-23.1-b48, mixed mode, sharing)

Quarkus version or git rev

3.16.3

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.9 (8e8579a9e76f7d015ee5ec7bfcdc97d260186937) Maven home: /usr/local/Cellar/maven/3.9.9/libexec Java version: 21.0.5, vendor: Azul Systems, Inc., runtime: /Library/Java/JavaVirtualMachines/zulu-21.jdk/Contents/Home Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "15.2", arch: "x86_64", family: "mac"

Additional information

No response

[quarkus-bot]

/cc @Karm (mandrel), @brunobat (opentelemetry), @galderz (mandrel), @quarkusio/devtools (maven), @radcortez (opentelemetry), @zakkak (mandrel,native-image)

[brunobat]

Seems not to find a lib:

Could not find agent library `$HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so in absolute path, with error: $HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so: cannot open shared object file: No such file or directory

According to the Dynatrace documentation, additional configs are needed to download the agent: https://docs.dynatrace.com/docs/ingest-from/technology-support/application-software/java/graalvm-native-image#maven-plugin
Are you performing all the necessary configs?

[viswatejan]

I have specified all the required configuration. I downloaded the agent and configured the path via the agentZip configuration property in the plugin.

I forgot to mention that I can successfully build the native image with buildpack and also with maven by directly pointing to my local GraalVM. It is just failing when I build using the Quarkus container build.

[zakkak]

@viswatejan as you rightly noticed this is happening because the container build doesn't have access to $HOME/Work/Projects/quarkus-project/target/tmp/graalnative/buildtime/lib64/liboneagentgraalnativebuildtime.so. I believe the best way to handle this would be to make sure the agent is fetched as pom dependency, this way Quarkus will place a copy of it in the lib directory it creates and passes to the classpath when building the native executable.

[viswatejan]

@zakkak, Thanks for your response. I will check with Dynatrace team and see if they can make their plugin work with the setup that you described.

[harrisonhenri]

Hi everyone,

Me and my teammates are facing the same problem. @viswatejan did you find any solution for this matter?

[brunobat]

@viswatejan, did @zakkak suggestion worked for you?

[viswatejan]

To fix this the way @zakkak suggested, it will be a change on the Dynatrace Maven plugin and I guess that should be requested using a feature request.

For now, I started to build against the local GraalVM instead of the container build.