Merge pull request #82619 from iblancasa/TRACING-4562

TRACING-4562: Improve Prometheus port and services documentation

Author: JoeAldinger

modules/otel-config-send-metrics-monitoring-stack.adoc

@@ -6,12 +6,7 @@
 [id="configuration-for-sending-metrics-to-the-monitoring-stack_{context}"]
 = Configuration for sending metrics to the monitoring stack
 
-One of two following custom resources (CR) configures the sending of metrics to the monitoring stack:
-
-* OpenTelemetry Collector CR
-* Prometheus `PodMonitor` CR
-
-A configured OpenTelemetry Collector CR can create a Prometheus `ServiceMonitor` CR for scraping the Collector's pipeline metrics and the enabled Prometheus exporters.
+You can configure the `OpenTelemetryCollector` custom resource (CR) to create a Prometheus `ServiceMonitor` CR or a `PodMonitor` CR for a sidecar deployment. A `ServiceMonitor` can scrape Collector's internal metrics endpoint and Prometheus exporter metrics endpoints.
 
 .Example of the OpenTelemetry Collector CR with the Prometheus exporter
 [source,yaml]
@@ -38,7 +33,15 @@ spec:
           receivers: [otlp]
           exporters: [prometheus]
 ----
-<1> Configures the Operator to create the Prometheus `ServiceMonitor` CR to scrape the Collector's internal metrics endpoint and Prometheus exporter metric endpoints. The metrics will be stored in the OpenShift monitoring stack.
+<1> Configures the {OTELOperator} to create the Prometheus `ServiceMonitor` CR or `PodMonitor` CR to scrape the Collector's internal metrics endpoint and the Prometheus exporter metrics endpoints.
+
+[NOTE]
+====
+Setting `enableMetrics` to `true` creates the following two `ServiceMonitor` instances:
+
+* One `ServiceMonitor` instance for the `<instance_name>-collector-monitoring` service. This `ServiceMonitor` instance scrapes the Collector's internal metrics.
+* One `ServiceMonitor` instance for the `<instance_name>-collector` service. This `ServiceMonitor` instance scrapes the metrics exposed by the Prometheus exporter instances.
+====
 
 Alternatively, a manually created Prometheus `PodMonitor` CR can provide fine control, for example removing duplicated labels added during Prometheus scraping.
 
@@ -71,4 +74,4 @@ spec:
 ----
 <1> The name of the OpenTelemetry Collector CR.
 <2> The name of the internal metrics port for the OpenTelemetry Collector. This port name is always `metrics`.
-<3> The name of the Prometheus exporter port for the OpenTelemetry Collector.
+<3> The name of the Prometheus exporter port for the OpenTelemetry Collector.

observability/otel/otel-collector/otel-collector-exporters.adoc

@@ -153,41 +153,42 @@ include::snippets/technology-preview.adoc[]
 [source,yaml]
 ----
 # ...
-  ports:
-  - name: promexporter # <1>
-    port: 8889
-    protocol: TCP
   config: |
     exporters:
       prometheus:
-        endpoint: 0.0.0.0:8889 # <2>
-        tls: # <3>
+        endpoint: 0.0.0.0:8889 # <1>
+        tls: # <2>
           ca_file: ca.pem
           cert_file: cert.pem
           key_file: key.pem
-        namespace: prefix # <4>
-        const_labels: # <5>
+        namespace: prefix # <3>
+        const_labels: # <4>
           label1: value1
-        enable_open_metrics: true # <6>
-        resource_to_telemetry_conversion: # <7>
+        enable_open_metrics: true # <5>
+        resource_to_telemetry_conversion: # <6>
           enabled: true
-        metric_expiration: 180m # <8>
-        add_metric_suffixes: false # <9>
+        metric_expiration: 180m # <7>
+        add_metric_suffixes: false # <8>
     service:
       pipelines:
         metrics:
           exporters: [prometheus]
 # ...
 ----
-<1> Exposes the Prometheus port from the Collector pod and service. You can enable scraping of metrics by Prometheus by using the port name in `ServiceMonitor` or `PodMonitor` custom resource.
-<2> The network endpoint where the metrics are exposed.
-<3> The server-side TLS configuration. Defines paths to TLS certificates.
-<4> If set, exports metrics under the provided value. No default.
-<5> Key-value pair labels that are applied for every exported metric. No default.
-<6> If `true`, metrics are exported using the OpenMetrics format. Exemplars are only exported in the OpenMetrics format and only for histogram and monotonic sum metrics such as `counter`. Disabled by default.
-<7> If `enabled` is `true`, all the resource attributes are converted to metric labels by default. Disabled by default.
-<8> Defines how long metrics are exposed without updates. The default is `5m`.
-<9> Adds the metrics types and units suffixes. Must be disabled if the monitor tab in Jaeger console is enabled. The default is `true`.
+<1> The network endpoint where the metrics are exposed. The {OTELOperator} automatically exposes the port specified in the `endpoint` field to the `<instance_name>-collector` service.
+<2> The server-side TLS configuration. Defines paths to TLS certificates.
+<3> If set, exports metrics under the provided value.
+<4> Key-value pair labels that are applied for every exported metric.
+<5> If `true`, metrics are exported by using the OpenMetrics format. Exemplars are only exported in the OpenMetrics format and only for histogram and monotonic sum metrics such as `counter`. Disabled by default.
+<6> If `enabled` is `true`, all the resource attributes are converted to metric labels. Disabled by default.
+<7> Defines how long metrics are exposed without updates. The default is `5m`.
+<8> Adds the metrics types and units suffixes. Must be disabled if the monitor tab in the Jaeger console is enabled. The default is `true`.
+
+
+[NOTE]
+====
+When the `spec.observability.metrics.enableMetrics` field in the `OpenTelemetryCollector` custom resource (CR) is set to `true`, the `OpenTelemetryCollector` CR automatically creates a Prometheus `ServiceMonitor` or `PodMonitor` CR to enable Prometheus to scrape your metrics.
+====
 
 [id="prometheus-remote-write-exporter_{context}"]
 == Prometheus Remote Write Exporter
@@ -276,4 +277,4 @@ include::snippets/technology-preview.adoc[]
 [role="_additional-resources"]
 [id="additional-resources_otel-collector-exporters_{context}"]
 == Additional resources
-* link:https://opentelemetry.io/docs/specs/otlp/[OpenTelemetry Protocol (OTLP) documentation]
+* link:https://opentelemetry.io/docs/specs/otlp/[OpenTelemetry Protocol (OTLP) documentation]

observability/otel/otel-configuring-otelcol-metrics.adoc

@@ -4,11 +4,22 @@
 include::_attributes/common-attributes.adoc[]
 :context: otel-configuring-metrics
 
-//[id="setting-up-monitoring-for-otel"]
-//== Setting up monitoring for the {OTELShortName}
-//The {OTELOperator} supports monitoring and alerting of each OpenTelemtry Collector instance and exposes upgrade and operational metrics about the Operator itself.
+The following list shows some of these metrics:
 
-You can enable metrics and alerts of OpenTelemetry Collector instances.
+	* Collector memory usage
+	* CPU utilization
+	* Number of active traces and spans processed
+	* Dropped spans, logs, or metrics
+	* Exporter and receiver statistics
+
+The {OTELOperator} automatically creates a service named `<instance_name>-collector-monitoring` that exposes the Collector's internal metrics. This service listens on port `8888` by default.
+
+You can use these metrics for monitoring the Collector's performance, resource consumption, and other internal behaviors. You can also use a Prometheus instance or another monitoring tool to scrape these metrics from the mentioned `<instance_name>-collector-monitoring` service.
+
+[NOTE]
+====
+When the `spec.observability.metrics.enableMetrics` field in the `OpenTelemetryCollector` custom resource (CR) is set to `true`, the `OpenTelemetryCollector` CR automatically creates a Prometheus `ServiceMonitor` or `PodMonitor` CR to enable Prometheus to scrape your metrics.
+====
 
 .Prerequisites
 
@@ -34,7 +45,9 @@ spec:
 
 You can use the *Administrator* view of the web console to verify successful configuration:
 
-* Go to *Observe* -> *Targets*, filter by *Source: User*, and check that the *ServiceMonitors* in the `opentelemetry-collector-<instance_name>` format have the *Up* status.
+. Go to *Observe* -> *Targets*.
+. Filter by *Source: User*.
+. Check that the *ServiceMonitors* or *PodMonitors* in the `opentelemetry-collector-<instance_name>` format have the *Up* status.
 
 .Additional resources
-* xref:../../observability/monitoring/enabling-monitoring-for-user-defined-projects.adoc#enabling-monitoring-for-user-defined-projects[Enabling monitoring for user-defined projects]
+* xref:../../observability/monitoring/enabling-monitoring-for-user-defined-projects.adoc#enabling-monitoring-for-user-defined-projects[Enabling monitoring for user-defined projects]