Oracle UCP 11 metrics not emitted on Prometheus exporter

[sirish-bajpai]

Hi,
I have a really simple Java application using Oracle UCP11 as connection pool and JDBC datasource, running on Azure AKS Kubernetes.

Oracle Jdbc and Ucp library versions are:

com.oracle.database.jdbc:ucp11:23.5.0.24.07
com.oracle.database.jdbc:ojdbc11:23.5.0.24.07

Using otel java agent ver 1.33.5 with metrics exporter set to Prometheus.
But neither Jdbc or Ucp pool metrics are showing up on the scrape page.
I have tried setting following flags, but nothing seems to make a difference:

otel.instrumentation.common.default-enabled=true
otel.instrumentation.oracle-ucp.enabled=true
otel.instrumentation.jdbc.enabled=true
otel.instrumentation.jdbc-datasource.enabled=true

Here is the output of scrape:

# HELP http_client_duration_milliseconds The duration of the outbound HTTP request
# TYPE http_client_duration_milliseconds histogram
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="0.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="5.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="10.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="25.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="50.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="75.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="100.0"} 0
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="250.0"} 9
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="500.0"} 12
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="750.0"} 12
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="1000.0"} 12
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="2500.0"} 12
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="5000.0"} 13
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="7500.0"} 13
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="10000.0"} 13
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha",le="+Inf"} 13
http_client_duration_milliseconds_count{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha"} 13
http_client_duration_milliseconds_sum{http_method="POST",http_status_code="201",net_peer_name="api.twilio.com",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="io.opentelemetry.apache-httpclient-4.0",otel_scope_version="1.33.5-alpha"} 5583.801020000001
# HELP process_runtime_jvm_buffer_count The number of buffers in the pool
# TYPE process_runtime_jvm_buffer_count gauge
process_runtime_jvm_buffer_count{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="direct"} 4.0
process_runtime_jvm_buffer_count{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="mapped"} 0.0
# HELP process_runtime_jvm_buffer_limit_bytes Total capacity of the buffers in this pool
# TYPE process_runtime_jvm_buffer_limit_bytes gauge
process_runtime_jvm_buffer_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="direct"} 32768.0
process_runtime_jvm_buffer_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="mapped"} 0.0
# HELP process_runtime_jvm_buffer_usage_bytes Memory that the Java virtual machine is using for this buffer pool
# TYPE process_runtime_jvm_buffer_usage_bytes gauge
process_runtime_jvm_buffer_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="direct"} 32768.0
process_runtime_jvm_buffer_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="mapped"} 0.0
# HELP process_runtime_jvm_classes_current_loaded Number of classes currently loaded
# TYPE process_runtime_jvm_classes_current_loaded gauge
process_runtime_jvm_classes_current_loaded{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 10891.0
# HELP process_runtime_jvm_classes_loaded_total Number of classes loaded since JVM start
# TYPE process_runtime_jvm_classes_loaded_total counter
process_runtime_jvm_classes_loaded_total{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 12491.0
# HELP process_runtime_jvm_classes_unloaded_total Number of classes unloaded since JVM start
# TYPE process_runtime_jvm_classes_unloaded_total counter
process_runtime_jvm_classes_unloaded_total{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 1600.0
# HELP process_runtime_jvm_cpu_utilization_ratio Recent cpu utilization for the process
# TYPE process_runtime_jvm_cpu_utilization_ratio gauge
process_runtime_jvm_cpu_utilization_ratio{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 0.01893939393939394
# HELP process_runtime_jvm_gc_duration_seconds Duration of JVM garbage collection actions
# TYPE process_runtime_jvm_gc_duration_seconds histogram
process_runtime_jvm_gc_duration_seconds_bucket{action="end of minor GC",gc="G1 Young Generation",otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",le="+Inf"} 1255
process_runtime_jvm_gc_duration_seconds_count{action="end of minor GC",gc="G1 Young Generation",otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 1255
process_runtime_jvm_gc_duration_seconds_sum{action="end of minor GC",gc="G1 Young Generation",otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 5.204999999999914
# HELP process_runtime_jvm_memory_committed_bytes Measure of memory committed
# TYPE process_runtime_jvm_memory_committed_bytes gauge
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-nmethods'",type="non_heap"} 2555904.0
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-profiled nmethods'",type="non_heap"} 8978432.0
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'profiled nmethods'",type="non_heap"} 2.5952256E7
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Compressed Class Space",type="non_heap"} 8912896.0
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Eden Space",type="heap"} 5242880.0
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Old Gen",type="heap"} 5.24288E7
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Survivor Space",type="heap"} 1048576.0
process_runtime_jvm_memory_committed_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Metaspace",type="non_heap"} 7.1303168E7
# HELP process_runtime_jvm_memory_init_bytes Measure of initial memory requested
# TYPE process_runtime_jvm_memory_init_bytes gauge
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-nmethods'",type="non_heap"} 2555904.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-profiled nmethods'",type="non_heap"} 2555904.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'profiled nmethods'",type="non_heap"} 2555904.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Compressed Class Space",type="non_heap"} 0.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Eden Space",type="heap"} 4194304.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Old Gen",type="heap"} 4194304.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Survivor Space",type="heap"} 0.0
process_runtime_jvm_memory_init_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Metaspace",type="non_heap"} 0.0
# HELP process_runtime_jvm_memory_limit_bytes Measure of max obtainable memory
# TYPE process_runtime_jvm_memory_limit_bytes gauge
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-nmethods'",type="non_heap"} 5828608.0
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-profiled nmethods'",type="non_heap"} 1.22916864E8
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'profiled nmethods'",type="non_heap"} 1.22912768E8
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Compressed Class Space",type="non_heap"} 1.12758784E8
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Old Gen",type="heap"} 7.5497472E7
process_runtime_jvm_memory_limit_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Metaspace",type="non_heap"} 1.21147392E8
# HELP process_runtime_jvm_memory_usage_after_last_gc_bytes Measure of memory used after the most recent garbage collection event on this pool
# TYPE process_runtime_jvm_memory_usage_after_last_gc_bytes gauge
process_runtime_jvm_memory_usage_after_last_gc_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Eden Space",type="heap"} 0.0
process_runtime_jvm_memory_usage_after_last_gc_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Old Gen",type="heap"} 3.2742376E7
process_runtime_jvm_memory_usage_after_last_gc_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Survivor Space",type="heap"} 1048576.0
# HELP process_runtime_jvm_memory_usage_bytes Measure of memory used
# TYPE process_runtime_jvm_memory_usage_bytes gauge
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-nmethods'",type="non_heap"} 1496704.0
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'non-profiled nmethods'",type="non_heap"} 8759296.0
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="CodeHeap 'profiled nmethods'",type="non_heap"} 2.312256E7
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Compressed Class Space",type="non_heap"} 8127256.0
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Eden Space",type="heap"} 2097152.0
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Old Gen",type="heap"} 4.3655328E7
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="G1 Survivor Space",type="heap"} 1048576.0
process_runtime_jvm_memory_usage_bytes{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha",pool="Metaspace",type="non_heap"} 6.9229832E7
# HELP process_runtime_jvm_system_cpu_load_1m Average CPU load of the whole system for the last minute
# TYPE process_runtime_jvm_system_cpu_load_1m gauge
process_runtime_jvm_system_cpu_load_1m{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 0.11962890625
# HELP process_runtime_jvm_system_cpu_utilization_ratio Recent cpu utilization for the whole system
# TYPE process_runtime_jvm_system_cpu_utilization_ratio gauge
process_runtime_jvm_system_cpu_utilization_ratio{otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 0.018933333333333333
# HELP process_runtime_jvm_threads_count Number of executing threads
# TYPE process_runtime_jvm_threads_count gauge
process_runtime_jvm_threads_count{daemon="false",otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 5.0
process_runtime_jvm_threads_count{daemon="true",otel_scope_name="io.opentelemetry.runtime-telemetry-java8",otel_scope_version="1.33.5-alpha"} 21.0
# TYPE target_info gauge
target_info{host_arch="amd64",host_name="***",os_description="Linux 5.15.0-1068-azure",os_type="linux",process_command_args="[/layers/paketo-buildpacks_adoptium/jre/bin/java, ***]",process_executable_path="/layers/paketo-buildpacks_adoptium/jre/bin/java",process_pid="1",process_runtime_description="Eclipse Adoptium OpenJDK 64-Bit Server VM 11.0.23+9",process_runtime_name="OpenJDK Runtime Environment",process_runtime_version="11.0.23+9",service_instance_id="659e429f-17f9-46f6-a463-2d031df90783",service_name="***",telemetry_auto_version="1.33.5",telemetry_sdk_language="java",telemetry_sdk_name="opentelemetry",telemetry_sdk_version="1.40.0"} 1

Do I need to setup something more for db metrics to get emitted on prometheus ?

[laurit]

Hopefully fixed with #12052

[sirish-bajpai]

Thanks !