Unable to send the traces to gateway url using auto instrumentation - giving Required SETTINGS preface not received when i use GRPC.

[ramanjaneyagupta]

Hi I am trying to send the traces to gateway( a set of collectors running behind the VIP), and when i am trying to send using GRPC it is giving below error -
ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter-Failedtoexportspans.Therequestcouldnotbe executed. Full error message: Required SETTINGS preface not received.
After a minute it is giving this :
ERROR io.opentelemetry.exporter.internal.grpc.OkhttpGrpcExporter-Failedtoexportspans.
The request could not be executed. Full error message: Broken pipe (Write failed)

Configuration:
java -javaagent:/home/rpasumar/agent/opentelemetry-javaagent_1.24.jar
-Dotel. traces.exporter=otlp
-Dotel.exporter.otlp.traces.endpoint=:4317
-Dotel.exporter.otlp.traces.headers-"tIs.insecure=false"
-Dotel.service.name=petclinic
-Dserver.port=8081
-jar target/spring-petclinic-2.6.0-SNAPSHOT.jar

NOTE: if i have the collector(agent mode) installed on same vm and send the traces to that local collector, from local collector to gateway i didnt see any issue.but directly sending it to gateway it is giving above issue.

[trask]

hi @ramanjaneyagupta!

The error message you are seeing suggests that there is a problem with the GRPC communication between your application and the collectors running behind the VIP. Specifically, the error message "Required SETTINGS preface not received" indicates that the communication between the client and server is not being established properly.

Here are a few things you can try to resolve the issue:

• Verify that the endpoint address is correct and that the collectors are running and reachable from your application.
• Check that your firewall settings allow traffic on the port you are using for communication with the collectors.
• Increase the logging level to get more detailed information about the issue. You can add the following to your application's command line arguments to enable debug logging: -Dio.opentelemetry.javaagent.slf4j.simpleLogger.defaultLogLevel=debug. This will enable debug logging for the OpenTelemetry Java agent, which may provide more information about the issue.

[ramanjaneyagupta]

and on collector side in logs it is saying in logs " received bogus greeting from the client"

[trask]

given that it's working with local collector, it seems like the Java agent is working correctly, and that it's some difference in the environment or setup of the remote collector

you could also try asking for advice in the collector repo

[ramanjaneyagupta]

@trask is there any way that we can send tIs.insecure, tls.insecure_skip_verify from the java agent ?

[trask]

otel.exporter.otlp.traces.endpoint=https://host:port will use tls

otel.exporter.otlp.traces.endpoint=http://host:port will not use tls

[ramanjaneyagupta]

Ok.. may be my question is a deviation.. even though my gateway cluster have tls enabled from my local collector on exporter option i am sending tls.insecure_skip_verify=true so that it is skipping the security while sending (or handshaking) now i am trying to send the traces directly to the gateway is there any option that we can send "tIs.insecure=false, tls.insecure_skip_verify=true" these values from the agent? thanks!