Springboot sqs instrumentation

[chenlin2001]

I want to instrument my springboot application with OpenTelemetry. When using the opentelemetry springboot starter I don't see support for SQS. Is there plan to support it? Or does it mean I have to not use the starter but use an agent?

[jaydeluca]

The java agent will provide that tracing, or alternatively, you can continue using the starter and then also use the aws library (docs here)

[laurit]

@chenlin2001 did you test this with localstack or actual aws?

[chenlin2001]

@laurit
localstack. However - I have the same test concept working with localsatck with micronaut sqs instrumentation so I know the issue is not localstack...
In addition I updated my example project to have the traceparent header in the listener code, so I can see it's received corrcetly
(so instrumenting the SqsTemplate for sending works), but the code that should be picking the context from the headers doesn't work. I'm attaching the updated example.
Do you want me to create a bug?
Uploading spring-otel-sqs-issues-updatedJuly9th.zip…

Logs with "Traceparent header: "

2025-07-09T08:28:48.241Z INFO 'Initializing Spring DispatcherServlet 'dispatcherServlet'' : e2ecb7c96c12388b4b654008322312dc 0ca6d13909d2cfaf [scopeInfo: org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/]:] {}
INFO|2025-07-09T09:28:48,243|{http-nio-auto-1-exec-1}|[org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/]]|e2ecb7c96c12388b4b654008322312dc|01|0ca6d13909d2cfaf|||||||| Initializing Spring DispatcherServlet 'dispatcherServlet'
2025-07-09T08:28:48.243Z INFO 'Initializing Servlet 'dispatcherServlet'' : e2ecb7c96c12388b4b654008322312dc 0ca6d13909d2cfaf [scopeInfo: org.springframework.web.servlet.DispatcherServlet:] {}
INFO|2025-07-09T09:28:48,243|{http-nio-auto-1-exec-1}|[org.springframework.web.servlet.DispatcherServlet]|e2ecb7c96c12388b4b654008322312dc|01|0ca6d13909d2cfaf|||||||| Initializing Servlet 'dispatcherServlet'
2025-07-09T08:28:48.244Z INFO 'Completed initialization in 1 ms' : e2ecb7c96c12388b4b654008322312dc 0ca6d13909d2cfaf [scopeInfo: org.springframework.web.servlet.DispatcherServlet:] {}
INFO|2025-07-09T09:28:48,244|{http-nio-auto-1-exec-1}|[org.springframework.web.servlet.DispatcherServlet]|e2ecb7c96c12388b4b654008322312dc|01|0ca6d13909d2cfaf|||||||| Completed initialization in 1 ms
[otel.javaagent 2025-07-09 09:28:48:313 +0100] [sdk-async-response-0-1] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'Sqs.GetQueueUrl' : e2ecb7c96c12388b4b654008322312dc c871d71c38ec8783 CLIENT [tracer: io.opentelemetry.aws-sdk-2.2:2.17.0-alpha] AttributesMap{data={aws.queue.name=test-sqs-queue-name, http.request.method=POST, rpc.method=GetQueueUrl, server.address=127.0.0.1, aws.request_id=620885f4-5320-4380-819a-5b2462841834, thread.id=54, url.full=http://127.0.0.1:35519/, http.response.status_code=200, aws.agent=java-aws-sdk, server.port=35519, rpc.service=Sqs, rpc.system=aws-api, thread.name=http-nio-auto-1-exec-1}, capacity=128, totalAddedValues=13}
[otel.javaagent 2025-07-09 09:28:48:330 +0100] [sdk-async-response-0-2] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'test-sqs-queue-name publish' : e2ecb7c96c12388b4b654008322312dc fe055de8d101c2c6 PRODUCER [tracer: io.opentelemetry.aws-sdk-2.2:2.17.0-alpha] AttributesMap{data={http.request.method=POST, rpc.method=SendMessage, server.address=127.0.0.1, aws.request_id=8e6279d5-7fd8-47b9-a70d-d5e79d2e6b37, thread.id=104, url.full=http://127.0.0.1:35519/, http.response.status_code=200, aws.agent=java-aws-sdk, messaging.message.id=957cf97c-3025-4bf9-a376-c339ee7ab520, server.port=35519, rpc.service=Sqs, messaging.system=aws_sqs, rpc.system=aws-api, thread.name=sdk-async-response-0-1, messaging.destination.name=test-sqs-queue-name, messaging.operation=publish, aws.queue.url=http://sqs.us-east-1.localhost:4566/000000000000/test-sqs-queue-name}, capacity=128, totalAddedValues=17}
2025-07-09T08:28:48.33Z INFO 'Message sent to SQS queue: GenericMessage [payload={"id":1,"message":"message"}, headers={myAttribute=Chen, id=c81acb5d-eee7-acd4-3a57-1307d7b89ad8, timestamp=1752049728291}]' : e2ecb7c96c12388b4b654008322312dc 0ca6d13909d2cfaf [scopeInfo: com.mimecast.architech.springk8sample.service.otel.OtelSqsExampleService:] {}
[otel.javaagent 2025-07-09 09:28:48:330 +0100] [sdk-async-response-0-3] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'test-sqs-queue-name receive' : 5541ea654fecf71252211ca05acda4ca e3557925b3367aee CONSUMER [tracer: io.opentelemetry.aws-sdk-2.2:2.17.0-alpha] AttributesMap{data={http.request.method=POST, rpc.method=ReceiveMessage, server.address=127.0.0.1, thread.id=109, url.full=http://127.0.0.1:35519/, http.response.status_code=200, aws.agent=java-aws-sdk, server.port=35519, rpc.service=Sqs, messaging.batch.message_count=1, messaging.system=aws_sqs, rpc.system=aws-api, thread.name=sdk-async-response-0-3, messaging.destination.name=test-sqs-queue-name, messaging.operation=receive}, capacity=128, totalAddedValues=15}
INFO|2025-07-09T09:28:48,330|{http-nio-auto-1-exec-1}|[com.mimecast.architech.springk8sample.service.otel.OtelSqsExampleService]|e2ecb7c96c12388b4b654008322312dc|01|0ca6d13909d2cfaf|||||||| Message sent to SQS queue: GenericMessage [payload={"id":1,"message":"message"}, headers={myAttribute=Chen, id=c81acb5d-eee7-acd4-3a57-1307d7b89ad8, timestamp=1752049728291}]
[otel.javaagent 2025-07-09 09:28:48:343 +0100] [http-nio-auto-1-exec-1] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'GET /otel-sqs/example' : e2ecb7c96c12388b4b654008322312dc 0ca6d13909d2cfaf SERVER [tracer: io.opentelemetry.tomcat-10.0:2.17.0-alpha] AttributesMap{data={http.request.method=GET, http.route=/otel-sqs/example, server.address=localhost, client.address=127.0.0.1, network.peer.address=127.0.0.1, url.path=/otel-sqs/example, thread.id=54, http.response.status_code=200, server.port=64848, network.protocol.version=1.1, user_agent.original=Apache-HttpClient/4.5.13 (Java/21.0.6), url.scheme=http, thread.name=http-nio-auto-1-exec-1, network.peer.port=64851}, capacity=128, totalAddedValues=14}
[otel.javaagent 2025-07-09 09:28:48:348 +0100] [main] INFO io.opentelemetry.exporter.logging.LoggingSpanExporter - 'GET' : e2ecb7c96c12388b4b654008322312dc 4f1a786f6dd51303 CLIENT [tracer: io.opentelemetry.apache-httpclient-4.0:2.17.0-alpha] AttributesMap{data={http.request.method=GET, network.protocol.version=1.1, server.address=localhost, thread.id=1, thread.name=main, url.full=http://localhost:64848/otel-sqs/example, http.response.status_code=200, server.port=64848}, capacity=128, totalAddedValues=8}
2025-07-09T08:28:48.361Z INFO 'Received message: SqsMessage(id=1, message=message)' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
INFO|2025-07-09T09:28:48,361|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| Received message: SqsMessage(id=1, message=message)
2025-07-09T08:28:48.361Z INFO 'Traceparent header: 00-e2ecb7c96c12388b4b654008322312dc-fe055de8d101c2c6-01' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
INFO|2025-07-09T09:28:48,361|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| Traceparent header: 00-e2ecb7c96c12388b4b654008322312dc-fe055de8d101c2c6-01
2025-07-09T08:28:48.361Z INFO 'Custom attribute myAttribute: Chen' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
INFO|2025-07-09T09:28:48,362|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| Custom attribute myAttribute: Chen
2025-07-09T08:28:48.362Z INFO 'Traceparent is present: 00-e2ecb7c96c12388b4b654008322312dc-fe055de8d101c2c6-01' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
INFO|2025-07-09T09:28:48,362|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| Traceparent is present: 00-e2ecb7c96c12388b4b654008322312dc-fe055de8d101c2c6-01
2025-07-09T08:28:48.362Z INFO 'myAttribute is present: Chen' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
INFO|2025-07-09T09:28:48,362|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| myAttribute is present: Chen
2025-07-09T08:28:48.362Z ERROR 'No valid OpenTelemetry span found in the current context' : 00000000000000000000000000000000 0000000000000000 [scopeInfo: com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample:] {}
ERROR|2025-07-09T09:28:48,362|{io.awspring.cloud.sqs.sqsListenerEndpointContainer#0-1}|[com.mimecast.architech.springk8sample.service.aws.sqs.SqsMessageListenerExample]||||||||||| No valid OpenTelemetry span found in the current context

[laurit]

localstack. However - I have the same test concept working with localsatck with micronaut sqs instrumentation so I know the issue is not localstack...

What you should keep in mind is that by default we use AWSTraceHeader messaging system attribute to propagate the context which is not supported in the free version of the localstack. I believe that otel.instrumentation.aws-sdk.experimental-use-propagator-for-messaging=true that you are already using is important to get it working on localstack.

In addition I updated my example project to have the traceparent header in the listener code, so I can see it's received corrcetly
(so instrumenting the SqsTemplate for sending works), but the code that should be picking the context from the headers doesn't work.

You are setting otel.instrumentation.messaging.experimental.receive-telemetry.enabled=true. With this flag producer and consumer will be in different traces, these traces are connected with span links. Perhaps this is what is confusing you?

I'm attaching the updated example.
Do you want me to create a bug?
Uploading spring-otel-sqs-issues-updatedJuly9th.zip…

Link to the latest sample does not work. I looked at the one you uploaded before and it does look like a bug. It would be easier if your test application included a description on what you expect us to do with it. Currently most of the tests it has fail. Idk maybe didn't run it correctly.

[laurit]

Hopefully resolved with #14207

[chenlin2001]

Sorry about the problems with previous versions. I'm attaching an updated project with Readme.md explaining the issue and how to see it in the test. None of the tests should fail at the moment

• Can you explain the expected behaviour you? I was expecting that same as in REST API if a traceparent arrives in the header the traceId will be used and a new span will be created, stating it as a valid span but with in the same trace. Why would producer/consumer be a different behviour with different traces? If I can see the traceparent available shouldn't it be used?
• But in any case no span is available at all - I'm getting to this point:

   if (span == null || !span.getSpanContext().isValid()) {
            log.error("No valid OpenTelemetry span found in the current context");

Uploading spring-otel-sqs-issues-extra-details.zip…

[chenlin2001]

I seem to be having issues uploading the project so created a repo for it: https://github.com/chenlin2001/spring-otel-sqs-issues

[chenlin2001]

I see 14207 is merged. Is it included in the latest release?
Will you be able to help me understand what should be set on:
otel.instrumentation.aws-sdk.experimental-use-propagator-for-messaging? Will setting it to true results as producer and consumer will be in different traces? If I want them to be with the same trace id should it be false?

[laurit]

I see #14207 is merged. Is it included in the latest release?

yes

Will you be able to help me understand what should be set on:
otel.instrumentation.aws-sdk.experimental-use-propagator-for-messaging? Will setting it to true results as producer and consumer will be in different traces? If I want them to be with the same trace id should it be false?

That property does not affect whether consumer and producer are in the same or linked traces. By default context propagation uses AWSTraceHeader message system attribute. Setting this property uses regular message attributes for context propagation.

[chenlin2001]

Thanks for the fast response.
So if the traceparent attribute is available as an attribute in the listener (as I saw it was with previous agent) will it creates a span from this attribute (using the trace id and flags value from the traceparent) once I use the new agent?

[chenlin2001]

I tested with the latest version of the agent (opentelemetry-javaagent - version: 2.18.1), and currently the span context is valid. However the trace id coming from the span context is not the same trace id as available in the traceparent attribute (which is the same value that is used for sending message). So we have 2 traces.

1. Is that expected behaviour?
2. Is that an issue with using localstack and not aws?
3. Is it possible to have the same trace id for the producer and consumer?

I updated the github project ( https://github.com/chenlin2001/spring-otel-sqs-issues) with the example

[laurit]

In https://github.com/chenlin2001/spring-otel-sqs-issues/blob/8ed50e43ca2a7cb43bd3b4c94126a30830c894f2/opentelemetry-javaagent.properties#L11 you set otel.instrumentation.messaging.experimental.receive-telemetry.enabled=true With this flag consumer and producer will be in different traces that are linked with a span link.

[chenlin2001]

Thank you, that was indeed the issue.