[awslogsencodingextension] Enhance CloudTrail log unmarshaler by adding missing fields and supporting Digest file content

[Kavindu-Dodan]

Component(s)

extension/awslogs_encoding

Is your feature request related to a problem? Please describe.

The current CloudTrail logs unmarshaler ¹ supports parsing a fair amount of fields from CloudTrail logs. However, several fields can further enhance the parsed log message. I have noticed the following,

• apiVersion : "Identifies the API version associated with the AwsApiCall eventType value" ²
• userIdentity.sessionContext : Extended details about user identity & session ³
• userIdentity.invokedBy : "The invokedBy field in the userIdentity element identifies the AWS service that made the API call" ³
• additionalEventData : May get omitted ⁴ but when present convey insights such as console sign in details ⁵

Along with the above, extension must add support for parsing of the Digest file structure ⁶. Digest files enable the validation of file integrity and are valuable for monitoring systems to interpret ingested data.

As a minor remark, it would be nice to preserve the TLSVersion as is without extracting the version part ⁷

Describe the solution you'd like

The improvements can be delivered in multiple PRs,

• Include missing fields & preserve TLS version as is
• Add support for CloudTrail Digest file format

Describe alternatives you've considered

None - Parsing and adding missing attributes is necessary.

Additional context

I am willing to contribute on this effort.

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

Footnotes

1. https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/encoding/awslogsencodingextension/internal/unmarshaler/cloudtraillog/unmarshaler.go#L57-L82 ↩

2. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-record-contents.html ↩

3. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-user-identity.html ↩ ↩²

4. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-record-contents.html#context-event-truncation-order ↩

5. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-event-reference-aws-console-sign-in-events.html ↩

6. https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-log-file-validation-digest-file-structure.html ↩

7. https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/encoding/awslogsencodingextension/internal/unmarshaler/cloudtraillog/unmarshaler.go#L227-L230 ↩

[github-actions]

Pinging code owners:

• extension/encoding/awslogsencoding: @axw @constanca-m

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[constanca-m]

Thanks @Kavindu-Dodan Let's move forward with this, I will assign it to you.

As a minor remark, it would be nice to preserve the TLSVersion as is without extracting the version part

Let's keep following semantic conventions. See.

[Kavindu-Dodan]

@constanca-m thank you, I am actively working on this to get it into the next release milestone.

[Kavindu-Dodan]

Update

• Include missing fields & preserve TLS version as is: Will be completed with the PR feat: enhance CloudTrail log encoder with extra attributes #43584
• Will start working on the CloudTrail Digest file format

[Kavindu-Dodan]

Update

PR #43737 adds digest file handling support and completes this issue.