How to instrument elasticsearch client 7.0 using OpenTelemetry

[venkatesh62891]

Hi Team,
I'm trying to manually instrument Elasticsearch client using OTEL. However, I see only java agent projects present in instrumentation folder but not any library folder which can be used in application and not through javaagent. Kindly let me know how to go about this requirement, appreciate your time.

[mateuszrzeszutek]

Hey @venkatesh62891 ,

You can start by taking a look at what the javaagent instrumentation is doing and trying to replicate it in your own code.
We don't have any library instrumentation for ES client because it does not expose any listener/interceptor types that would allow us to inject the instrumentation code into the client's behavior; which is why we have to do the bytecode instrumentation.

[venkatesh62891]

@mateuszrzeszutek I have tried the option to extend RestClient using the same package but still RestClientBuilder is another challenge which will create instance of only RestClient. Both RestClient and builder are pieced together and restricted to find a way to override the behavior.
Solution-1

1. Create wrapper for RestClient and extend RestClient in same package as RestClient.
2. Create a wrapper for RestClientBuilder and modify code to return RestClientWrapper instead of RestClient(RestClientBuilder need not and cannot be extended as it is final).
3. Build RestClientWrapper using RestClientBuilderWrapper and inject into ElasticSearchClient.
4. Do create and end span around super.performRequest(Request) with required span parameters same as java agent for Elasticsearch.

Solution-2
Since I'm using a spring boot application, I'm using AspectJAutoProxy

1. Create an Aspect around the method RestClient.performRequest.
2. Do create and end span around joinpoint.proceed with required span parameters same as java agent for Elasticsearch.

Both will work just fine but be cautious with Solution-1. Since the RestClientBuilder cannot be extended, a wrapper for builder has to added to the code alternate to RestClientBuilder class. Original Builder class can be modified by SDK providers and we may always have to sequence the update with every release version.

[mateuszrzeszutek]

Since I'm using a spring boot application, I'm using AspectJAutoProxy

That's probably the better choice here 👍
Using AspectJ, in this case you should be able to achieve very similar results as with the javaagent.

[laurit]

I created a pr with library instrumentation for elasticsearch rest client #8911

[venkatesh62891]

Hi @laurit , thank you for taking time to create the solution for Elasticsearch instrumentation. When can we expect the PR to be merged?

[trask]

Likely before the August release

[venkatesh62891]

• We are currently working on native OTel instrumentation for the ES client, however, this will be only available for new client versions (probably 8.10+). With that, the instrumentation will be built directly into the client library and won't require bytecode instrumentation anymore.

@AlexanderWert Currently I'm using Elasticsearch client 8.7. When is the native client instrumentation be expected to hit the market?

[AlexanderWert]

Hi @venkatesh62891 ,
sorry for the delayed reply.

We are currently working on that, so it should be available with one of the next releases of the client (NOT before version 8.10).