Merge pull request #95529 from openshift-cherrypick-robot/cherry-pick-95436-to-enterprise-4.17

[enterprise-4.17] OBSDOCS-1330: Add the metrics refrence table for remote write

Author: stevsmit

modules/monitoring-table-of-remote-write-metrics.adoc

@@ -0,0 +1,23 @@
+// Module included in the following assemblies:
+//
+// * observability/monitoring/troubleshooting-monitoring-issues.adoc
+
+:_mod-docs-content-type: REFERENCE
+[id="table-of-remote-write-metrics_{context}"]
+= Table of remote write metrics
+
+The following table contains remote write and remote write-adjacent metrics with further description to help solve issues during remote write configuration.
+
+[options="header"]
+|===
+| Metric | Description
+| `prometheus_remote_storage_highest_timestamp_in_seconds` | Shows the newest timestamp that Prometheus stored in the write-ahead log (WAL) for any sample.
+| `prometheus_remote_storage_queue_highest_sent_timestamp_seconds` | Shows the newest timestamp that the remote write queue successfully sent.
+| `prometheus_remote_storage_samples_retried_total` | The number of samples that remote write failed to send and had to resend to remote storage. A steady high rate for this metric indicates problems with the network or remote storage endpoint. 
+| `prometheus_remote_storage_shards` | Shows how many shards are currently running for each remote endpoint.
+| `prometheus_remote_storage_shards_desired` | Shows the calculated needed number of shards based on the current write throughput and the rate of incoming versus sent samples.
+| `prometheus_remote_storage_shards_max` | Shows the maximum number of shards based on the current configuration.
+| `prometheus_remote_storage_shards_min` | Shows the minimum number of shards based on the current configuration.
+| `prometheus_tsdb_wal_segment_current` | The WAL segment file that Prometheus is currently writing new data to.
+| `prometheus_wal_watcher_current_segment` | The WAL segment file that each remote write instance is currently reading from.
+|===

observability/monitoring/configuring-core-platform-monitoring/configuring-metrics.adoc

@@ -38,6 +38,8 @@ include::modules/monitoring-example-remote-write-queue-configuration.adoc[levelo
 * link:https://prometheus.io/docs/practices/remote_write/#remote-write-tuning[Remote write tuning (Prometheus documentation)]
 * xref:../../../nodes/pods/nodes-pods-secrets.adoc#nodes-pods-secrets-about_nodes-pods-secrets[Understanding secrets]
 
+include::modules/monitoring-table-of-remote-write-metrics.adoc[leveloffset=+2]
+
 //Creating cluster ID labels for metrics for core platform monitoring
 include::modules/monitoring-creating-cluster-id-labels-for-metrics.adoc[leveloffset=+1,tags=**;CPM;!UWM]
 

observability/monitoring/configuring-user-workload-monitoring/configuring-metrics-uwm.adoc

@@ -38,6 +38,8 @@ include::modules/monitoring-example-remote-write-queue-configuration.adoc[levelo
 * link:https://prometheus.io/docs/practices/remote_write/#remote-write-tuning[Remote write tuning (Prometheus documentation)]
 * xref:../../../nodes/pods/nodes-pods-secrets.adoc#nodes-pods-secrets-about_nodes-pods-secrets[Understanding secrets]
 
+include::modules/monitoring-table-of-remote-write-metrics.adoc[leveloffset=+2]
+
 // Creating cluster ID labels for metrics for monitoring of user-defined projects
 include::modules/monitoring-creating-cluster-id-labels-for-metrics.adoc[leveloffset=+1,tags=**;!CPM;UWM]