[Bug] skywalking swck inject bug #13241

hhhhuanzi · 2025-05-12T09:43:40Z

hhhhuanzi
May 12, 2025

Search before asking

I had searched in the issues and found no similar issues.

Apache SkyWalking Component

SWCK (apache/skywalking-swck)

What happened

SWCK injects the Java Agent into the /sky/agent directory of the container. However, it fails to start. If the injection is not used and the agent is manually downloaded and built into the container using docker build, the startup works fine.

Environment:

JDK Version: OpenJDK version "21-ea" 2023-09-19
SkyWalking Version: OAP 10.0.0, UI 10.0.0
SWCK Version: 0.9.0
Java Agent Version: 9.4.0-java21

The path of the swck injected agent inside the container: /sky/agent/skywalking-agent.jar
The path of the manually built agent inside the container: /sky2/skywalking-agent.jar

Error Log:

root@demo-6b9c9f9ffb-6fdjm:/app# java -javaagent:/sky/agent/skywalking-agent.jar -jar demo-0.0.1-SNAPSHOT.jar 
Picked up JAVA_TOOL_OPTIONS: 
DEBUG 2025-05-12 09:25:03.317 main AgentPackagePath : The beacon class location is jar:file:/sky/agent/skywalking-agent.jar!/org/apache/skywalking/apm/agent/core/boot/AgentPackagePath.class. 
INFO 2025-05-12 09:25:03.321 main SnifferConfigInitializer : Config file found in /sky/agent/config/agent.config. 

INFO 2025-05-12 09:25:07.305 main SkyWalkingAgent : Skywalking agent begin to install transformer ... 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::                (v3.4.5)

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:115)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:102)
        at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:64)
        at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:40)
Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.skywalking.apm.agent.core.meter.MeterService.register(org.apache.skywalking.apm.agent.core.meter.BaseMeter)" because "org.apache.skywalking.apm.agent.core.meter.AbstractBuilder.METER_SERVICE" is null
        at org.apache.skywalking.apm.agent.core.meter.AbstractBuilder.build(AbstractBuilder.java:73)
        at org.apache.skywalking.apm.agent.core.so11y.AgentSo11y.durationOfInterceptor(AgentSo11y.java:139)
        at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ConstructorInter.intercept(ConstructorInter.java:78)
        at ch.qos.logback.classic.spi.LoggingEvent.<init>(LoggingEvent.java)
        at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:424)
        at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:386)
        at ch.qos.logback.classic.Logger.log(Logger.java:780)
        at org.apache.commons.logging.LogAdapter$Slf4jLocationAwareLog.warn(LogAdapter.java:445)
        at org.springframework.boot.SpringApplication.handleRunFailure(SpringApplication.java:821)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:328)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1362)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1351)
        at com.example.wanghuandemo.WanghuanDemoApplication.main(WanghuanDemoApplication.java:10)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        ... 4 more

### Logs from starting with the manually injected agent:

root@demo-6b9c9f9ffb-6fdjm:/app# java -javaagent:/sky2/skywalking-agent.jar -jar demo-0.0.1-SNAPSHOT.jar 
Picked up JAVA_TOOL_OPTIONS: 
DEBUG 2025-05-12 09:25:03.317 main AgentPackagePath : The beacon class location is jar:file:/sky/agent/skywalking-agent.jar!/org/apache/skywalking/apm/agent/core/boot/AgentPackagePath.class. 
INFO 2025-05-12 09:25:03.321 main SnifferConfigInitializer : Config file found in /sky/agent/config/agent.config. 
INFO 2025-05-12 09:36:09.188 main SkyWalkingAgent : Skywalking agent begin to install transformer ... 

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::                (v3.4.5)

2025-05-12 09:36:40.390 [main] INFO  com.example.wanghuandemo.WanghuanDemoApplication [TID:N/A] - Starting WanghuanDemoApplication v0.0.1-SNAPSHOT using Java 21-ea with PID 257 (/app/wanghuan-demo-0.0.1-SNAPSHOT.jar started by root in /app)
2025-05-12 09:36:40.416 [main] INFO  com.example.wanghuandemo.WanghuanDemoApplication [TID:N/A] - No active profile set, falling back to 1 default profile: "default"
2025-05-12 09:36:51.086 [main] INFO  o.s.d.r.config.RepositoryConfigurationDelegate [TID:N/A] - Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2025-05-12 09:36:51.923 [main] INFO  o.s.d.r.config.RepositoryConfigurationDelegate [TID:N/A] - Finished Spring Data repository scanning in 703 ms. Found 1 JPA repository interface.
2025-05-12 09:36:59.706 [main] INFO  o.s.boot.web.embedded.tomcat.TomcatWebServer [TID:N/A] - Tomcat initialized with port 8080 (http)
2025-05-12 09:36:59.813 [main] INFO  org.apache.coyote.http11.Http11NioProtocol [TID:N/A] - Initializing ProtocolHandler ["http-nio-8080"]
2025-05-12 09:36:59.824 [main] INFO  org.apache.catalina.core.StandardService [TID:N/A] - Starting service [Tomcat]
2025-05-12 09:36:59.825 [main] INFO  org.apache.catalina.core.StandardEngine [TID:N/A] - Starting Servlet engine: [Apache Tomcat/10.1.40]

What you expected to happen

swck injected has bug

How to reproduce

1、download java-agent tar on official website。
2、use dockerfile command 'COPY' skywalking folder into container
3、use swck swagent to inject skywalking-java-agent into container
4、exec container，export JAVA_TOOLS_OPTS=''
5、use java -javaagent:/sky/agent/skywalking-agent.jar watch error log
6、use java -javaagent:/sky2/skywalking.jar watch success log。/sky2/skywalking.jar is docker build into container

Anything else

No response

Are you willing to submit a pull request to fix on your own?

Yes I am willing to submit a pull request on my own!

Code of Conduct

I agree to follow this project's Code of Conduct

wu-sheng · 2025-05-12T09:49:28Z

wu-sheng
May 12, 2025
Collaborator

@kezhenxu94 Any insight about this?

0 replies

hhhhuanzi · 2025-05-13T03:04:21Z

hhhhuanzi
May 13, 2025
Author

I found the reason for the SWCK injection startup failure: it was caused by a conflict between two optional plugins. The SWCK injection used the configuration of SWAgent, which included both apm-trace-ignore-plugin-9.4.0.jar and trace-sampler-cpu-policy-plugin-9.4.0.jar as optional plugins:

optionalPlugins:

trace-ignore
trace

The reason why manually building the Docker image allows successful startup is that the optional plugins were not copied into the plugin directory, so there was no conflict.

0 replies

kezhenxu94 · 2025-05-13T03:32:17Z

kezhenxu94
May 13, 2025
Collaborator

@hhhhuanzi make sure you provided useful information, at least, (1) provide the command/configurations of how you installed SWCK, (2) the configuration of your JavaAgent kubectl get javaagent -n skywalking-system -l app=demo -oyaml

0 replies

kezhenxu94 · 2025-05-13T03:38:07Z

kezhenxu94
May 13, 2025
Collaborator

Also, even if the optional plugins are enabled, the error is unexpected, I think there is a bug in the so11y, when registering meter in the plugins, the meter service is not yet initialized, FYI @weixiang1862

Caused by: java.lang.NullPointerException: Cannot invoke "org.apache.skywalking.apm.agent.core.meter.MeterService.register(org.apache.skywalking.apm.agent.core.meter.BaseMeter)" because "org.apache.skywalking.apm.agent.core.meter.AbstractBuilder.METER_SERVICE" is null
at org.apache.skywalking.apm.agent.core.meter.AbstractBuilder.build(AbstractBuilder.java:73)
at org.apache.skywalking.apm.agent.core.so11y.AgentSo11y.durationOfInterceptor(AgentSo11y.java:139)
at org.apache.skywalking.apm.agent.core.plugin.interceptor.enhance.ConstructorInter.intercept(ConstructorInter.java:78)

0 replies

hhhhuanzi · 2025-05-13T03:38:17Z

hhhhuanzi
May 13, 2025
Author

My container has successfully started now. My solution was to keep only the apm-trace-ignore-plugin-9.4.0.jar plugin and not use the trace-sampler-cpu-policy-plugin-9.4.0.jar.

1、install swck： kubectl apply -f config/operator-bundle.yaml
2、java agent config：
[root@k8s-node3 config]# kubectl get javaagent -n ng-i8 app-demo-javaagent -oyaml

apiVersion: operator.skywalking.apache.org/v1alpha1
kind: JavaAgent
metadata:
  creationTimestamp: "2025-05-12T10:47:40Z"
  generation: 1
  labels:
    app: demo
  name: app-demo-javaagent
  namespace: ng-i8
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: ReplicaSet
    name: demo-6b9c9f9ffb
    uid: f41ace11-da0d-4d56-8904-a231a6cbd747
  resourceVersion: "32321889"
  uid: f633e2be-275a-4be0-a48c-4534c348e1e8
spec:
  agentConfiguration:
    collector.backend_service: skywalking-oap.skywalking.svc.cluster.local:11800
  backendService: skywalking-oap.skywalking.svc.cluster.local:11800
  podSelector: app=demo
  serviceName: Your_ApplicationName
status:
  creationTime: "2025-05-12T10:47:40Z"
  expectedInjectiedNum: 4
  lastUpdateTime: "2025-05-13T03:25:04Z"
  realInjectedNum: 4

0 replies

wu-sheng · 2025-05-13T04:49:21Z

wu-sheng
May 13, 2025
Collaborator

The NPE should be able to be fixed by apache/skywalking-java#755.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Bug] skywalking swck inject bug #13241

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 6 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

[Bug] skywalking swck inject bug #13241

Uh oh!

Uh oh!

hhhhuanzi May 12, 2025

Search before asking

Apache SkyWalking Component

What happened

Environment:

Error Log:

What you expected to happen

How to reproduce

Anything else

Are you willing to submit a pull request to fix on your own?

Code of Conduct

Replies: 6 comments

Uh oh!

wu-sheng May 12, 2025 Collaborator

Uh oh!

hhhhuanzi May 13, 2025 Author

Uh oh!

kezhenxu94 May 13, 2025 Collaborator

Uh oh!

kezhenxu94 May 13, 2025 Collaborator

Uh oh!

Uh oh!

hhhhuanzi May 13, 2025 Author

Uh oh!

wu-sheng May 13, 2025 Collaborator

hhhhuanzi
May 12, 2025

wu-sheng
May 12, 2025
Collaborator

hhhhuanzi
May 13, 2025
Author

kezhenxu94
May 13, 2025
Collaborator

kezhenxu94
May 13, 2025
Collaborator

hhhhuanzi
May 13, 2025
Author

wu-sheng
May 13, 2025
Collaborator