Add service entity support for EMF exporter

[musa-asad]

Description of the issue

To support the Explore related feature in CloudWatch, the CloudWatch Agent sends an "Entity", which includes relevant metadata to correlate metrics or logs between resources (e.g., an EKS cluster) and services (e.g., a Java application). In the case of exporting custom metrics received by OTLP or from scraping Prometheus targets, we need to export a service "Entity" for EMF, which includes the following attributes: Service, Environment, EKS.Cluster/K8s.Cluster, K8s.Namespace, K8s.Workload, K8s.Node, AWS.ServiceNameSource, PlatformType, and EC2.InstanceId.

Right now, in the CloudWatch Agent's entity processor, we generate these "Entity" fields, but they're in the following format: com.amazonaws.cloudwatch.entity.internal.<field>. We then filter them out when creating labels in the EMF exporter.

We need to convert them into their expected format when exporting for EMF logs.

Description of changes

Important

Co-PRs: aws/amazon-cloudwatch-agent#1529; aws-observability/helm-charts#171

• Migrate current entity label filtering logic to filtering out dimensions as we don't want "Entity" fields to show up as dimensions.
• Convert entity attributes into expected entity fields when generating field map for EMF logs.
• Create entityattributes.go to hold entity constants and field retrieval logic.
• Add flag to enable entity.
• Add unit tests.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

CloudWatch Agent Logs

2025-02-06T17:27:40Z D! {"caller":"awsemfexporter@v0.103.0/emf_exporter.go:160","msg":"Finish processing resource metrics","kind":"exporter","data_type":"metrics","name":"awsemf","labels":{"com.amazonaws.cloudwatch.entity.internal.aws.account.id":"637423426239","com.amazonaws.cloudwatch.entity.internal.deployment.environment":"eks:entity-cluster-2/default","com.amazonaws.cloudwatch.entity.internal.instance.id":"i-0da7f196c5fa59a25","com.amazonaws.cloudwatch.entity.internal.k8s.cluster.name":"entity-cluster-2","com.amazonaws.cloudwatch.entity.internal.k8s.namespace.name":"default","com.amazonaws.cloudwatch.entity.internal.k8s.node.name":"ip-XXX-XX-XX-XX.us-west-2.compute.internal","com.amazonaws.cloudwatch.entity.internal.k8s.workload.name":"sample-app","com.amazonaws.cloudwatch.entity.internal.platform.type":"AWS::EKS","com.amazonaws.cloudwatch.entity.internal.service.name":"unknown_service:java","com.amazonaws.cloudwatch.entity.internal.service.name.source":"Instrumentation","com.amazonaws.cloudwatch.entity.internal.type":"Service","host.arch":"amd64","host.name":"sample-app-54d797bb9b-nkb77", "k8s.deployment.name":"sample-app","k8s.namespace.name":"default","k8s.node.name":"ip-XXX-XX-XX-XX.us-west-2.compute.internal","k8s.pod.ip":"XXX.XX.XX.XX","k8s.pod.name":"sample-app-54d797bb9b-nkb77","k8s.replicaset.name":"sample-app-54d797bb9b","os.description":"Linux 6.1.119-129.201.amzn2023.x86_64","os.type":"linux","process.command_line":"/usr/lib/jvm/java-17-openjdk-amd64:bin:java -javaagent:/aws-observability/classpath/aws-opentelemetry-agent-1.17.0.jar","process.executable.path":"/usr/lib/jvm/java-17-openjdk-amd64:bin:java","process.pid":"","process.runtime.description":"Debian OpenJDK 64-Bit Server VM 17.0.4+8-Debian-1deb11u1","process.runtime.name":"OpenJDK Runtime Environment","process.runtime.version":"17.0.4+8-Debian-1deb11u1","service.name":"unknown_service:java","telemetry.auto.version":"1.17.0-aws","telemetry.sdk.language":"java","telemetry.sdk.name":"opentelemetry","telemetry.sdk.version":"1.17.0"}}

EMF Log

{
    "AWS.ServiceNameSource": "Instrumentation",
    "CloudWatchMetrics": [
        {
            "Namespace": "CWAgent",
            "Dimensions": [
                [
                    "process.runtime.description",
                    "telemetry.sdk.language",
                    "telemetry.sdk.name",
                    "service.name",
                    "process.executable.path",
                    "telemetry.auto.version",
                    "k8s.pod.ip",
                    "k8s.node.name",
                    "k8s.pod.name",
                    "process.pid",
                    "os.description",
                    "telemetry.sdk.version",
                    "k8s.namespace.name",
                    "host.arch",
                    "process.runtime.version",
                    "host.name",
                    "process.runtime.name",
                    "os.type",
                    "k8s.deployment.name",
                    "process.command_line",
                    "k8s.replicaset.name"
                ]
            ],
            "Metrics": [
                {
                    "Name": "process.runtime.jvm.classes.unloaded"
                },
                {
                    "Name": "process.runtime.jvm.classes.loaded"
                },
                {
                    "Name": "process.runtime.jvm.classes.current_loaded"
                },
                {
                    "Name": "process.runtime.jvm.threads.count"
                }
            ]
        }
    ],
    "EC2.InstanceId": "i-0da7f196c5fa59a25",
    "Environment": "eks:entity-cluster-2/default",
    "EKS.Cluster": "entity-cluster-2",
    "K8s.Namespace": "default",
    "K8s.Node": "ip-XXX-XX-XX-XX.us-west-2.compute.internal",
    "K8s.Workload": "sample-app",
    "Service": "unknown_service:java",
    "PlatformType": "AWS::EKS",
    "Timestamp": "1738862978053",
    "Version": "0",
    "host.arch": "amd64",
    "host.name": "sample-app-54d797bb9b-nkb77",
    "k8s.deployment.name": "sample-app",
    "k8s.namespace.name": "default",
    "k8s.node.name": "ip-XXX-XX-XX-XX.us-west-2.compute.internal",
    "k8s.pod.ip": "XXX.XX.XX.XX",
    "k8s.pod.name": "sample-app-54d797bb9b-nkb77",
    "k8s.replicaset.name": "sample-app-54d797bb9b",
    "os.description": "Linux 6.1.119-129.201.amzn2023.x86_64",
    "os.type": "linux",
    "process.command_line": "/usr/lib/jvm/java-17-openjdk-amd64:bin:java -javaagent:/aws-observability/classpath/aws-opentelemetry-agent-1.17.0.jar",
    "process.executable.path": "/usr/lib/jvm/java-17-openjdk-amd64:bin:java",
    "process.pid": "1",
    "process.runtime.description": "Debian OpenJDK 64-Bit Server VM 17.0.4+8-Debian-1deb11u1",
    "process.runtime.name": "OpenJDK Runtime Environment",
    "process.runtime.version": "17.0.4+8-Debian-1deb11u1",
    "service.name": "unknown_service:java",
    "telemetry.auto.version": "1.17.0-aws",
    "telemetry.sdk.language": "java",
    "telemetry.sdk.name": "opentelemetry",
    "telemetry.sdk.version": "1.17.0",
    "process.runtime.jvm.classes.current_loaded": 0,
    "process.runtime.jvm.classes.loaded": 0,
    "process.runtime.jvm.classes.unloaded": 0,
    "process.runtime.jvm.threads.count": 0
}