Span rate dropped after moving to 2.x agent

[daniel-verygoodsecurity]

I'm not sure how to troubleshoot this, but developers have complained that the span rate dropped by over 60% after switching from the 1.x agent to 2.x. I'm currently using 2.15.0 and seeing a lot less spans come through. reverting back to 1.x is near impossible at the moment, even for testing.

I've tried setting OTEL_INSTRUMENTATION_EXPERIMENTAL_SPAN_SUPPRESSION_STRATEGY=none as discussed in #11636 (comment). I appear to be experiencing the same problem. The spans do not show up as root spans in Grafana/Tempo. I see a lot of <root span not yet received>.

[laurit]

2.x generates less span by default, see the breaking changes section in https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/tag/v2.0.0

I've tried setting OTEL_INSTRUMENTATION_EXPERIMENTAL_SPAN_SUPPRESSION_STRATEGY=none as discussed in #11636 (comment). I appear to be experiencing the same problem. The spans do not show up as root spans in Grafana/Tempo. I see a lot of <root span not yet received>.

Since currently you have not provided any details it is unlikely that we can help you based on the currently provided information. It would be ideal if you could provide us with a minimal application that reproduces your issue. I would guess that <root span not yet received> means that there is a span that refers to a parent span that is missing. I guess you could start your investigation from investigating what should be the parent span of the span you have and what instrumentation creates it. The discussion you linked is about a bug where spans from requests don't start a new traces but are added to an existing traces, creating a really large trace. Based on the provided info I'd say that you probably are experiencing a different issue.