Histogram View with custom buckets and custom exporter

[mohnishkodnani]

Hi,
I have a custom http server and via that I can create an endpoint where metrics would be exposed. I just want to expose the metrics in prometheus format. I followed the example in another discussion. However, in my case when I serialize this, I am not seeing the view attached to the histogram aggregation config take effect and wanted to see what i am doing wrong.

TEST_CASE("Metric API", "[metrics]") {
   // Initialize the registry
    metrics::otel_registry registry;
    SECTION("Test histogram metric observe() functionality") {
        auto buckets = metrics::buckets_info{0, 5, 10, 20, 50, 100}; // custom buckets
        auto histogram = registry.create_histogram("my_histogram", "my histogram to test", buckets);
        histogram->observe(10);
        histogram->observe(20);
        auto metrics_str = registry.export_metrics();
        REQUIRE_THAT(metrics_str, 
            Catch::Matchers::ContainsSubstring("# HELP my_histogram my histogram to test\n# TYPE my_histogram histogram\nmy_histogram_bucket{le=\"5\"} 0\nmy_histogram_bucket{le=\"10\"} 1\nmy_histogram_bucket{le=\"20\"} 2\nmy_histogram_bucket{le=\"50\"} 2\nmy_histogram_bucket{le=\"100\"} 2\nmy_histogram_bucket{le=\"+Inf\"} 2\nmy_histogram_sum 30\nmy_histogram_count 2"));
    }

Some classes
Create histogram function and export function

    std::unique_ptr<histogram> otel_registry::create_histogram(const std::string &name, const std::string &help, const buckets_info &buckets) {
        // Add custom buckets to the aggregation config.
        auto config = std::make_shared<opentelemetry::sdk::metrics::HistogramAggregationConfig>(); 
        config->boundaries_ = buckets;
        // Create a view to attach the custom buckets to the histogram.
        auto view = std::make_unique<opentelemetry::sdk::metrics::View>(name + "_view", help, "", opentelemetry::sdk::metrics::AggregationType::kHistogram, config);
        // Instrument selector to find the instrument to apply the view.
        auto instrument_selector = std::make_unique<opentelemetry::sdk::metrics::InstrumentSelector>(opentelemetry::sdk::metrics::InstrumentType::kHistogram, name, "");
        // Fetch the global MeterProvider
        auto meter_provider = std::static_pointer_cast<opentelemetry::sdk::metrics::MeterProvider>(opentelemetry::metrics::Provider::GetMeterProvider());
        // Get the meter Selector
        auto meter_selector = std::make_unique<opentelemetry::sdk::metrics::MeterSelector>(name, version, schema);
        // Add the view to the provider.
        meter_provider->AddView(std::move(instrument_selector), std::move(meter_selector), std::move(view));
        // Create the histogram instrument.
        auto hist = meter_->CreateDoubleHistogram(name, help);
        return std::make_unique<otel_histogram>(std::move(hist));
    }

    std::string otel_registry::export_metrics() {
        // auto provider = opentelemetry::metrics::Provider::GetMeterProvider();
        // auto sdk_provider = static_cast<opentelemetry::sdk::metrics::MeterProvider *>(provider.get());
        return cexporter_->Export();
    }

Custom exporter

    class custom_prometheus_exporter : public opentelemetry::sdk::metrics::MetricReader {        
        public:
            custom_prometheus_exporter(): collection_lock_(), serializer_() {};
            /**
             * Exports the data to a string representation of all metrics in prometheus format.
             *
             * @return A string representation of the exported data in prometheus format.
             */
            std::string Export() {
              std::ostringstream oss;
              if(!IsShutdown()) {
                Collect([&oss,
                         this](opentelemetry::sdk::metrics::ResourceMetrics
                                   &resource_metrics_) {
                  std::lock_guard<std::mutex> lock(collection_lock_);
                  auto prometheus_metric_data = opentelemetry::exporter::
                      metrics::PrometheusExporterUtils::TranslateToPrometheus(
                          resource_metrics_, true, true);
                  serializer_.Serialize(oss, prometheus_metric_data);
                  return true;
                });
              }
              return oss.str();
            }
            opentelemetry::sdk::metrics::AggregationTemporality GetAggregationTemporality(opentelemetry::sdk::metrics::InstrumentType it) const noexcept override {
                return opentelemetry::sdk::metrics::AggregationTemporality::kCumulative;
            }
            /**
             * \brief Forces a flush of the exporter with an optional timeout.
             *
             * This function is called to force a flush of the exporter. It can be used to ensure that all collected metrics
             * are sent to the Prometheus server. An optional timeout can be provided to limit the duration of the flush operation.
             *
             * \param timeout The timeout duration for the flush operation. By default, it is set to `std::chrono::microseconds::max`,
             *                indicating no timeout.
             * \return `true` if the flush operation was successful, `false` otherwise.
             */
            bool OnForceFlush(std::chrono::microseconds timeout = (std::chrono::microseconds::max)()) noexcept override {
                return true;
            }
            /**
             * @brief Handles the shutdown of the exporter.
             *
             * This function is called when the exporter is being shut down. It allows for a timeout
             * to be specified, after which the shutdown process will be forcefully terminated.
             *
             * @param timeout The timeout duration for the shutdown process. Defaults to the maximum
             *                possible duration if not specified.
             * @return `true` if the shutdown process completed successfully, `false` otherwise.
             */
            bool OnShutDown(std::chrono::microseconds timeout = (std::chrono::microseconds::max)()) noexcept override {
                return true;
            }
          private:
            mutable std::mutex collection_lock_;
            prometheus::TextSerializer serializer_;
    };

otel_registry class constructor

       otel_registry() {
           cexporter_ = std::make_shared<custom_prometheus_exporter>();
            auto ostr_exp = opentelemetry::exporter::metrics::OStreamMetricExporterFactory::Create();
            opentelemetry::sdk::metrics::PeriodicExportingMetricReaderOptions reader_options;
            reader_options.export_interval_millis = std::chrono::milliseconds(1000);
            reader_options.export_timeout_millis = std::chrono::milliseconds(500);
            auto reader = opentelemetry::sdk::metrics::PeriodicExportingMetricReaderFactory::Create(std::move(ostr_exp), reader_options);
             auto context = opentelemetry::sdk::metrics::MeterContextFactory::Create();
             context->AddMetricReader(std::move(reader));
             context->AddMetricReader(cexporter_);
            auto u_provider =
                opentelemetry::sdk::metrics::MeterProviderFactory::Create(std::move(context));
            auto meter_provider =
                static_cast<opentelemetry::sdk::metrics::MeterProvider *>(
                    u_provider.get());
            meter_ = static_cast<opentelemetry::sdk::metrics::Meter *>(u_provider->GetMeter(name, version, schema).get());
           opentelemetry::metrics::Provider::SetMeterProvider(std::move(u_provider));
        }

I see the default buckets but not the one I passed into the aggregation config.

"# HELP target Target metadata
  # TYPE target gauge
  target_info{service_name="unknown_service",telemetry_sdk_language="cpp",
  telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.14.2"} 1
  1714672322750
  # HELP my_histogram my histogram to test
  # TYPE my_histogram histogram
  my_histogram_count 2 1714672322750
  my_histogram_sum 30 1714672322750
  my_histogram_bucket{le="0"} 0 1714672322750
  my_histogram_bucket{le="5"} 0 1714672322750
  my_histogram_bucket{le="10"} 1 1714672322750
  my_histogram_bucket{le="25"} 2 1714672322750
  my_histogram_bucket{le="50"} 2 1714672322750
  my_histogram_bucket{le="75"} 2 1714672322750
  my_histogram_bucket{le="100"} 2 1714672322750
  my_histogram_bucket{le="250"} 2 1714672322750
  my_histogram_bucket{le="500"} 2 1714672322750
  my_histogram_bucket{le="750"} 2 1714672322750
  my_histogram_bucket{le="1000"} 2 1714672322750
  my_histogram_bucket{le="2500"} 2 1714672322750
  my_histogram_bucket{le="5000"} 2 1714672322750
  my_histogram_bucket{le="7500"} 2 1714672322750
  my_histogram_bucket{le="10000"} 2 1714672322750
  my_histogram_bucket{le="+Inf"} 2 1714672322750

[lalitb]

@mohnishkodnani There are unit-tests for validating custom buckets -

opentelemetry-cpp/sdk/test/metrics/histogram_test.cc

Line 193 in 2d077f8

TEST(Histogram, UInt64CustomBuckets)

. If that helps troubleshoot?

[mohnishkodnani]

I have followed those tests to create this code that I have pasted above and as you can see the string I get back has all the default buckets and not the view that I have setup. I want to make sure that when I eventually use my custom exporter, it can take into account the View.
I cannot find code that when exporting take the View into account or how the view interacts with the Histogram that is created using the Meter.

[mohnishkodnani]

I modified the custom_prometheus_exporter Export function to do what the unit test does and I still get 15 boundary buckets and not the 6 that I am trying to pass in the View.

            std::string Export() {
              std::ostringstream oss;
              if(!IsShutdown()) {
                Collect([&oss,
                         this](opentelemetry::sdk::metrics::ResourceMetrics
                                   &resource_metrics_) {
                  std::lock_guard<std::mutex> lock(collection_lock_);
                  for (const opentelemetry::sdk::metrics::ScopeMetrics &smd : resource_metrics_.scope_metric_data_) {
                    for (const opentelemetry::sdk::metrics::MetricData &md : smd.metric_data_) {
                      for (const opentelemetry::sdk::metrics::PointDataAttributes &dp : md.point_data_attr_) {
                        if(md.instrument_descriptor.type_ == opentelemetry::v1::sdk::metrics::InstrumentType::kHistogram) {
                          auto d = opentelemetry::nostd::get<
                              opentelemetry::sdk::metrics::HistogramPointData>(
                              dp.point_data);
                          std::ostringstream o;
                          for (double b : d.boundaries_) {
                            o << b << ",";
                          }
                          std::cout << o.str() << std::endl;
                        }
                      }
                    }
                  }
                  auto prometheus_metric_data = opentelemetry::exporter::
                      metrics::PrometheusExporterUtils::TranslateToPrometheus(
                          resource_metrics_, true, true);
                  serializer_.Serialize(oss, prometheus_metric_data);
                  return true;
                });
              }
              return oss.str();
            }

[lalitb]

I cannot find code that when exporting take the View into account or how the view interacts with the Histogram that is created using the Meter.

It is happening here - https://github.com/open-telemetry/opentelemetry-cpp/blob/main/sdk/src/metrics/meter.cc#L306. Look for FindViews call to find the matching view for the given instrument. Ensure that the correct instrument information is provided while creating the view. Else, provide a compilable repro sample - I can try debug.

[mohnishkodnani]

Provided a repro sample below.

[mohnishkodnani]

@lalitb see if this helps.
https://github.com/mohnishkodnani/opentelemetry-cpp-wrapper/blob/main/opentelemetry/src/main.cpp

Build Commands

❯ cmake -DCMAKE_BUILD_TYPE:STRING=Debug -S<PATH>/opentelemetry-cpp-wrapper -B<PATH>/opentelemetry-cpp-wrapper/build_try -GNinja

> cmake --build .

When I run the main program.

❯ ./metrics_opentelemetry_exe 
0,5,10,25,50,75,100,250,500,750,1000,2500,5000,7500,10000,
# HELP target Target metadata
# TYPE target gauge
target_info{service_name="unknown_service",telemetry_sdk_language="cpp",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.15.0"} 1 1715143666951
# HELP my_histogram my histogram to test
# TYPE my_histogram histogram
my_histogram_count 2 1715143666951
my_histogram_sum 30 1715143666951
my_histogram_bucket{le="0"} 0 1715143666951
my_histogram_bucket{le="5"} 0 1715143666951
my_histogram_bucket{le="10"} 1 1715143666951
my_histogram_bucket{le="25"} 2 1715143666951
my_histogram_bucket{le="50"} 2 1715143666951
my_histogram_bucket{le="75"} 2 1715143666951
my_histogram_bucket{le="100"} 2 1715143666951
my_histogram_bucket{le="250"} 2 1715143666951
my_histogram_bucket{le="500"} 2 1715143666951
my_histogram_bucket{le="750"} 2 1715143666951
my_histogram_bucket{le="1000"} 2 1715143666951
my_histogram_bucket{le="2500"} 2 1715143666951
my_histogram_bucket{le="5000"} 2 1715143666951
my_histogram_bucket{le="7500"} 2 1715143666951
my_histogram_bucket{le="10000"} 2 1715143666951
my_histogram_bucket{le="+Inf"} 2 1715143666951

[mohnishkodnani]

Found a problem in meter_selector ( variable shadow was creating it with the wrong name.