Update Collector Telemetry Metric Collection syntax in ECS-EC2 integration

opentelemetry/ecs-ec2-windows/CHANGELOG.md

@@ -5,5 +5,10 @@
 <!-- ### version / full date -->
 <!-- * [Update/Bug fix] message that describes the changes that you apply -->
 
+
+### 0.0.2 / 2025-3-15
+* Added Collector metric collection to ECS-EC2-Windows example
+* Updated doc to remove deprecated S3 config and Base64 env var features
+
 ### 0.0.1 / 2023-09-11
 * Added EC2 ECS Windows Example for metrics collection

opentelemetry/ecs-ec2-windows/README.md

@@ -40,8 +40,8 @@ For Windows, this receiver does not support being run as a daemonset, as such, e
 
 ### Open Telemetry Configuration
 
-The Open Telemetry configuration is embedded in this cloudformation template by default, however, you do have the option of specifying your own configuration my modifying the template. The  The Coralogix Open Telemetry distribution supports reading configration from S3 as well as an Envrionmentat Variable. Note that Environment variables can be raw strings or Base64 encoded. Configuration from S3 must be passed to the collector using the S3 URL of the object, for example `cdot --config s3://{your-bucket}.s3.{region}.amazonaws.com/{your-object-key}`, when using this feature in ECS, the host or task must have sufficient permissions to read S3 Objects.
+The Open Telemetry configuration is embedded in this CloudFormation template by default. However, you have the option of specifying your own configuration by modifying the template. The Coralogix Open Telemetry distribution supports reading configuration over HTTP as well as an Environment Variable. Note that Environment Variables must be raw strings. Base64 encoding is not supported.
 
 ### Image
 
-This example uses the [coralogixrepo/coralogix-otel-collector:0.1.0-windowsserver-1809](https://hub.docker.com/layers/coralogixrepo/coralogix-otel-collector/0.1.0-windowsserver-1809/images/sha256-c436b2b29501592449e2b72a2393f4825e7216bdd62d90cb5e14463a46fafd95?context=explore) image which is a custom distribution of Open Telemetry containing custom components developed by Coralogix. The image is available on [Docker Hub](https://hub.docker.com/r/coralogixrepo/coralogix-otel-collector). The ECS components are described [here](../ecs-ec2/components.md)
+This example uses the [coralogixrepo/coralogix-otel-collector:0.4.1-windowsserver-1809](https://hub.docker.com/layers/coralogixrepo/coralogix-otel-collector/0.4.1-windowsserver-1809/images/sha256-c436b2b29501592449e2b72a2393f4825e7216bdd62d90cb5e14463a46fafd95?context=explore) image which is a custom distribution of Open Telemetry containing custom components developed by Coralogix. The image is available on [Docker Hub](https://hub.docker.com/r/coralogixrepo/coralogix-otel-collector). The ECS components are described [here](../ecs-ec2/components.md)

opentelemetry/ecs-ec2-windows/template.yaml

@@ -141,7 +141,33 @@ Resources:
                     awsecscontainermetricsd:
                       sidecar: true
 
+                    prometheus:
+                      config:
+                        scrape_configs:
+                          - job_name: otel-collector-metrics
+                            scrape_interval: 30s
+                            static_configs:
+                              - targets: ["localhost:8888"]
+
                   processors:
+                    # remove unneeded labels from metrics added as of otel v0.119.0
+                    transform/prometheus:
+                      error_mode: ignore
+                      metric_statements:
+                        - context: metric
+                          statements:
+                            - replace_pattern(name, "_total$", "")
+                        - context: datapoint
+                          statements:
+                            - delete_key(attributes, "otel_scope_name") where resource.attributes["service.name"] == "cdot"
+                            - delete_key(attributes, "service.name") where resource.attributes["service.name"] == "cdot"
+
+                        - context: resource
+                          statements:
+                            - delete_key(attributes, "service.instance.id") where attributes["service.name"] == "cdot"
+                            - delete_key(attributes, "service.name") where attributes["service.name"] == "cdot"
+
+
                     batch:
 
                   exporters:
@@ -161,16 +187,31 @@ Resources:
                       timeout: 30s
 
                   service:
+                    telemetry:
+                      logs:
+                        level: warn
+
+                      metrics:
+                        readers:
+                          - pull:
+                              exporter:
+                                prometheus:
+                                  host: 0.0.0.0
+                                  port: 8888
+
                     pipelines:
                       metrics:
                         receivers:
                           - awsecscontainermetricsd
+                          - prometheus
                         processors:
+                          - transform/prometheus
                           - batch
                         exporters:
                           - coralogix
 
 
+        ### --- your app task definition goes here --- ###
         - Name: "win-app"
           Image: !Ref AppImage
           Essential: true

opentelemetry/ecs-ec2/CHANGELOG.md

@@ -5,6 +5,11 @@
 <!-- ### version / full date -->
 <!-- * [Update/Bug fix] message that describes the changes that you apply -->
 
+### 1.0.1 / 2025-03-15
+- Updated default otel config for ECS-EC2 to use new otel collector metric syntax
+- Added transform to remove unneeded labels from metrics added as of otel v0.119.0
+
+
 ### 1.0.0 / 2025-01-14
 ### 🛑 Breaking changes 🛑
 - Adjusted embeded configuration to support logs/metrics and traces by default.

opentelemetry/ecs-ec2/template.yaml

@@ -203,6 +203,24 @@ Mappings:
                 statements:
                   - keep_keys(attributes, [""])
 
+          # remove unneeded labels from metrics added as of otel v0.119.0
+          transform/prometheus:
+            error_mode: ignore
+            metric_statements:
+              - context: metric
+                statements:
+                  - replace_pattern(name, "_total$", "")
+              - context: datapoint
+                statements:
+                  - delete_key(attributes, "otel_scope_name") where resource.attributes["service.name"] == "cdot"
+                  - delete_key(attributes, "service.name") where resource.attributes["service.name"] == "cdot"
+
+              - context: resource
+                statements:
+                  - delete_key(attributes, "service.instance.id") where attributes["service.name"] == "cdot"
+                  - delete_key(attributes, "service.name") where attributes["service.name"] == "cdot"
+
+
           batch: { send_batch_max_size: 2048, send_batch_size: 1024, timeout: 1s }
 
           resource/metadata:
@@ -314,15 +332,20 @@ Mappings:
 
             metrics/otel-collector:
               receivers: [prometheus, hostmetrics]
-              processors: [resourcedetection/otel-collector, resource/metadata, batch]
+              processors: [resourcedetection/otel-collector, resource/metadata, transform/prometheus, batch]
               exporters: [coralogix]
 
           telemetry:
             logs:
               level: warn
+
             metrics:
-              address: 0.0.0.0:8888
-              level: detailed
+              readers:
+                - pull:
+                    exporter:
+                      prometheus:
+                        host: 0.0.0.0
+                        port: 8888
 
   CoralogixRegionMap:
     EU1: