[Bug] The traceId get from logstash are all mixed up accross multiple tomcat threads

[whohasthis]

Search before asking

• I had searched in the issues and found no similar issues.

Apache SkyWalking Component

Java Agent (apache/skywalking-java)

What happened

We use TraceIdJsonProvider of apm-toolkit-logback-1.x to add the trace id into the logstash fields sent to ELK, we are expecting to use the trace id to get all related log message throughout all services that the trace id involves.

Below is the logback config part related to the trace id logstash appender

logstash.:5045


{
"service" : {
"category" : "xxx",
"type" : "xxxxx",
"name" : "xxx-xxx-xx"
}
}

But somehow the trace ids are all mixed up, we are seeing that one trace id associated to many different tomcat thread name in the same short period of time, below are part of the result we found on ELK

What you expected to happen

We expect to get one trace id per one thread name during the same period of time, not all mixed up accoss many different thread names.

How to reproduce

We use skywalking 8.4.0, OS centos 6.7,
Skywalking OAP server is running on K8S env, using ES 6.7 as the storage.
The service is running on tomcat 8.5.65 with spring MVC framework.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[wu-sheng]

If trace id gets mixed, it means you are facing some plugin side issue. You need to locate what it is, and why(not supported version, not supported cast).

[wu-sheng]

Tomcat plugin has little chance to have this issue. Its multiple threads never reused before the request finished.