[MLCommons] Add details about metrics integration (#10120)

* feat: add details about metrics framework in ml-commons

Signed-off-by: Pavan Yekbote <pybot@amazon.com>

* Doc review

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Fix links

Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>

* Update _monitoring-your-cluster/metrics/getting-started.md

Signed-off-by: Nathan Bower <nbower@amazon.com>

---------

Signed-off-by: Pavan Yekbote <pybot@amazon.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
Signed-off-by: Nathan Bower <nbower@amazon.com>
Co-authored-by: Fanit Kolchina <kolchfa@amazon.com>
Co-authored-by: Nathan Bower <nbower@amazon.com>

Author: 3 people

_ml-commons-plugin/api/stats.md

@@ -7,6 +7,9 @@ nav_order: 110
 
 # Stats
 
+The Stats API provides basic statistics about ML Commons, such as the number of running tasks. To monitor machine learning workflows using more detailed time-series metrics, see [Monitoring machine learning workflows]({{site.url}}{{site.baseurl}}/monitoring-your-cluster/metrics/getting-started/#monitoring-machine-learning-workflows).
+{: .note }
+
 Gets statistics related to the number of tasks. 
 
 ## Endpoints

_monitoring-your-cluster/metrics/getting-started.md

@@ -35,11 +35,11 @@ The `enable` flag is toggled using a Java Virtual Machine (JVM) parameter that i
   cd \path\to\opensearch
   ```
 
-2. Open your `opensearch.yaml` file.
-3. Add the following setting to `opensearch.yaml`:
+2. Open your `opensearch.yml` file.
+3. Add the following setting to `opensearch.yml`:
 
-  ```bash
-  opensearch.experimental.feature.telemetry.enabled=true
+  ```yaml
+  opensearch.experimental.feature.telemetry.enabled: true
   ```
   {% include copy.html %}
 
@@ -73,7 +73,7 @@ export OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.telemetry.enabled
 
 ### Enable with Docker 
 
-If you’re running OpenSearch using Docker, add the following line to `docker-compose.yml` under `environment`:
+If you're running OpenSearch using Docker, add the following line to `docker-compose.yml` under `environment`:
 
 ```bash
 OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.telemetry.enabled=true"
@@ -105,3 +105,17 @@ The metrics framework feature supports the following metric types:
 2. **UpDown counters:** UpDown counters can be incremented with positive values or decremented with negative values. UpDown counters are well suited for tracking metrics like open connections, active requests, and other fluctuating quantities.
 3. **Histograms:** Histograms are valuable tools for visualizing the distribution of continuous data. Histograms offer insight into the central tendency, spread, skewness, and potential outliers that might exist in your metrics. Patterns such as normal distribution, skewed distribution, or bimodal distribution can be readily identified, making histograms ideal for analyzing latency metrics and assessing percentiles.
 4. **Asynchronous Gauges:** Asynchronous gauges capture the current value at the moment a metric is read. These metrics are non-additive and are commonly used to measure CPU utilization on a per-minute basis, memory utilization, and other real-time values.
+
+## Monitoring machine learning workflows
+Introduced 3.1
+{: .label .label-purple }
+
+OpenSearch provides enhanced observability for [machine learning (ML)]({{site.url}}{{site.baseurl}}/ml-commons-plugin/) workflows. Metrics related to ML operations are pushed directly to the core metrics registry, giving you improved visibility into model usage and performance. Additionally, every 5 minutes, a periodic job collects and exports state data, helping you monitor the health and activity of your ML workloads over time. 
+
+To enable ML observability, specify the following settings in `opensearch.yml`:
+
+```yaml
+plugins.ml_commons.metrics_collection_enabled: true
+plugins.ml_commons.metrics_static_collection_enabled: true
+```
+{% include copy.html %}