Transformation of metric data to OTLP data should be a public method

[mayank-yadav2]

Recently, I am designing a framework where we will instrument the data. Then the data has to be converted to OTLP data. Save the data in file. Finally, another service will read the OTLP data from the file and send it to another gateway. So, I want to perform these steps :

• Instrument the data using instrumentation counters.
• As and when the counter is updated, read the data using manual reader. This returns data in sdk.metricdata.ResourceMetrics format.
• Now convert this sdk data into OTLP format data. (This is where there is no public method available to do it. More on this below.)
• Save the OTLP data into a file
• Read this data from the file.

Problem faced :
Now, I see that there is otlpgrpcexporter available that does two things :

1. It transforms the sdk data into OTLP format.
2. It also sends this OTLP data over grpc to the assigned destination.

But I only want to transform the data and not send it. This functionality of only transforming the data into OTLP format is not public. It is inside internal folder, hence making it a private method.

Reasons for it to be a public method :
There are multiple reasons why transforming data into OTLP format should be public :

• The whole point of OTEL is to give you the data in OTLP format. User should have that functionality available where he/she can convert its custom data into OTLP format. (Custom data here being sdk data)
• There is already a public method available to read the data and save it in sdk.metricdata.ResourceMetrics. So, if there is an option to convert your instrumented data into sdk format, why is there no option to convert sdk data into OTLP format. There should also be an option to either collect data in OTLP format or convert sdk data into OTLP format. Since, a lot of users finally want OTLP data from this package.
• This would be helpful for the third-party exporters also. Any developer who is wanting to export data in OTLP format won't have to write its own logic. It should be readily available by this package itself.

[XSAM]

@mayank-yadav2 Thanks for creating this. I know it would be frustrating if you want to DIY something that is not in the regular pipeline.

I have one concern: the OTel spec does not mention publishing a public transform method for users to convert SDK data to OTLP. I am not sure we can add a new functionality without changing the spec.

I know this is not ideal, but an existing workaround for you is to leverage the fileexporter from the OpenTelemetry Collector to export metrics to files.

The data flow would be like this: instrument metrics in application -> export OTLP via gRPC/HTTP exporter -> OTel collector -> file exporter.

@open-telemetry/go-maintainers any thoughts?