Parser for ostream span exporter output

[gaborcsardi]

I am wondering if anybody knows a way to parse the output of the ostream span exporter. That would make it easier to test the SDK that I am implementing on top of opentelemetry-cpp.

The format seems pretty simple, but it is not exactly JSON or YAML.

[dbarker]

I had the same use case and tried to build tests around the ostream exporters but settled on implementing a custom exporter using the standard opentelemetry-proto protobuf messages. The advantage of this approach is that you have stability guarantees with the protobuf interface. On the flip side the ostream exporter output isn't defined in the spec, may not include all the data from otlp, and may change at any time.

Here is some code from a trace exporter that will call a callback on export with the opentelemetry::proto::collector::trace::v1::ExportTraceServiceRequest. The proto messages provide a c++ interface or you can use protobuf's json utilities if you need json.

#include <opentelemetry/nostd/span.h>
#include <opentelemetry/proto/collector/trace/v1/trace_service.pb.h>
#include <opentelemetry/sdk/common/exporter_utils.h>
#include <opentelemetry/sdk/trace/exporter.h>
#include <opentelemetry/sdk/trace/recordable.h>

#include <chrono>
#include <functional>
#include <memory>

namespace opentelemetry_extensions::exporters::otlp_proto
{
using ExportTraceServiceRequest = opentelemetry::proto::collector::trace::v1::ExportTraceServiceRequest;
using OtlpTraceProtoExportCallback = std::function<opentelemetry::sdk::common::ExportResult(ExportTraceServiceRequest)>;

class OtlpTraceProtoExporter final : public opentelemetry::sdk::trace::SpanExporter
{
public:
  OtlpTraceProtoExporter(OtlpTraceProtoExportCallback callback);

  std::unique_ptr<opentelemetry::sdk::trace::Recordable> MakeRecordable() noexcept override;

  opentelemetry::sdk::common::ExportResult
  Export(const opentelemetry::nostd::span<std::unique_ptr<opentelemetry::sdk::trace::Recordable>>& spans) noexcept
      override;

  bool ForceFlush(std::chrono::microseconds) noexcept override;
  bool Shutdown(std::chrono::microseconds) noexcept override;

private:
  OtlpTraceProtoExportCallback callback_;
};
}  // namespace opentelemetry_extensions::exporters::otlp_proto

#include "opentelemetry_extensions/exporters/otlp_trace_proto_exporter.hpp"

#include <google/protobuf/arena.h>
#include <opentelemetry/exporters/otlp/otlp_recordable.h>
#include <opentelemetry/exporters/otlp/otlp_recordable_utils.h>

#include <memory>

namespace opentelemetry_extensions::exporters::otlp_proto
{
std::unique_ptr<google::protobuf::Arena> create_trace_protobuf_arena()
{
  google::protobuf::ArenaOptions arena_options;
  arena_options.initial_block_size = 1024;
  arena_options.max_block_size = 65536;
  return std::make_unique<google::protobuf::Arena>(arena_options);
}

OtlpTraceProtoExporter::OtlpTraceProtoExporter(OtlpTraceProtoExportCallback callback) : callback_(std::move(callback))
{
}

std::unique_ptr<opentelemetry::sdk::trace::Recordable> OtlpTraceProtoExporter::MakeRecordable() noexcept
{
  return std::make_unique<opentelemetry::exporter::otlp::OtlpRecordable>();
}

opentelemetry::sdk::common::ExportResult OtlpTraceProtoExporter::Export(
    const opentelemetry::nostd::span<std::unique_ptr<opentelemetry::sdk::trace::Recordable>>& spans) noexcept
{
  if (spans.empty())
  {
    return opentelemetry::sdk::common::ExportResult::kSuccess;
  }

  auto arena = create_trace_protobuf_arena();

  auto* request = google::protobuf::Arena::Create<ExportTraceServiceRequest>(arena.get());

  opentelemetry::exporter::otlp::OtlpRecordableUtils::PopulateRequest(spans, request);

  return callback_(*request);
}

bool OtlpTraceProtoExporter::ForceFlush(std::chrono::microseconds) noexcept
{
  return true;
}

bool OtlpTraceProtoExporter::Shutdown(std::chrono::microseconds) noexcept
{
  return true;
}

}  // namespace opentelemetry_extensions::exporters::otlp_proto

[gaborcsardi]

Thanks! Yes, that definitely seems like a better way to go, maybe I'll do that as well. For now I wrote a rudimentary parser.

I guess at some point the file exporter [1] will be stable, probably, that'll probably simplify this further.

[1] https://opentelemetry.io/docs/specs/otel/protocol/file-exporter/

[lalitb]

The ostream span exporter format is not standard as per the specs, it can change, and so not meant used for validation. It would be better to use InMemoryExporters for any such validations.

[gaborcsardi]

Thanks! Changes are fine, parts of the spec are changing, anyway. I am vendoring opentelemetry-cpp, so whenever I update the vendored code I can update the tests as well.