Skip to content

Fix: Lambda Topology Issue #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Feb 5, 2025
Merged

Fix: Lambda Topology Issue #149

merged 1 commit into from
Feb 5, 2025

Conversation

yiyuan-he
Copy link
Contributor

@yiyuan-he yiyuan-he commented Feb 5, 2025 •
edited
Loading

Issue #, if available:
Lambda Topology Issue Context: aws-observability/aws-otel-python-instrumentation#319

Description of changes:

Test plan:

The same cases in the test plan of this PR were validated by building a custom JavaScript lambda layer. Below is a screenshot of all the test cases in a single topology.

We can observe the following:

  • Service entity node for Invoke call to downstream lambda.
  • AWS Resource node for GetFunction call to downstream lambda.
  • AWS Resource node for ListBuckets call to downstream s3.
  • Another AWS Resource node for GetBucketLocation call to downstream s3.
Screenshot 2025-02-05 at 10 15 26 AM

Here we see downstream lambda called with Invoke is correctly treated as RemoteService entity when not instrumented:
image

Additionally, I generated the spans locally to validate the new lambda instrumentation patch behaves correctly.

Generated span for Invoke call

We see the new aws.remote.environment attribute is present in span so the downstream lambda will be treated as a Service type in Application Signals backend.

invoke

Generated span for GetFunction call

We see aws.remote.resource.identifier and aws.cloudformation.primary.identifier are set so the downstream lambda will be treated as an AWS resource type in Application Signals backend.

get-function

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@yiyuan-he yiyuan-he requested a review from a team as a code owner February 5, 2025 00:40
@yiyuan-he yiyuan-he force-pushed the lambda-topology-fix branch 2 times, most recently from 2b7aa02 to fbd6454 Compare February 5, 2025 16:50
@yiyuan-he yiyuan-he changed the title lambda topology issue fix [WIP] Fix: Lambda Topology Issue Feb 5, 2025
@yiyuan-he yiyuan-he force-pushed the lambda-topology-fix branch from fbd6454 to 84b468d Compare February 5, 2025 18:11
@yiyuan-he yiyuan-he merged commit da8291e into aws-observability:main Feb 5, 2025
9 checks passed
This was referenced Feb 7, 2025
Merged
Merged
yiyuan-he added a commit to aws-observability/aws-otel-java-instrumentation that referenced this pull request Feb 11, 2025
@yiyuan-he
Fix: Lambda Topology Issue (#1016)
cd3c2c5 
**Issue #, if available:**

Lambda Topology issue -- more context in PRs for Python and JavaScript:
-
aws-observability/aws-otel-python-instrumentation#319
-
aws-observability/aws-otel-js-instrumentation#149

**Description of changes:**

- Apply fix for the Lambda Topology issue. The logic mimics the fix in
our other ADOT SDKs.
- Adding back AWS Resource support for Lambda.
-
#907
  - We previously removed support due to the Lambda Topology issue

**Test plan:**

Set up two Lambda functions with Java runtimes and tested with custom
Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are
screenshots of the topology for various configurations.

**v1 Topology (lambdaA & lambdaB instrumented)**
<img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM"
src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7"
/>

**v2 Topology (lambdaA & lambdaB instrumented)**
<img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM"
src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5"
/>

We observe the following correct behaviors for topology above:
- Service entity node for `Invoke` call to downstream lambda.
- AWS Resource node for `GetFunction` call to downstream lambda.
- AWS Resource node for `ListBuckets` call to downstream s3.

**v1 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

**v2 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

We observe the following correct behaviors for topology above:
- Downstream lambda called with `Invoke` is correctly treated as
RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda
instrumentation patch behaves correctly.

**v1 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM"
src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5"
/>

**v2 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM"
src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2"
/>

**v1 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM"
src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02"
/>

**v2 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM"
src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2"
/>

We observe the following correct behaviors in the spans above:
- For `Invoke` calls, we see `aws.remote.service` and
`aws.remote.environment` correctly populated in the spans.
- For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource
attributes such as `aws.remote.resource.identifier` and
`aws.cloudformation.primary.identifier` correctly populated.


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
yiyuan-he added a commit to yiyuan-he/aws-otel-java-instrumentation that referenced this pull request Mar 5, 2025
@yiyuan-he
Fix: Lambda Topology Issue (aws-observability#1016)
224134d 
**Issue #, if available:**

Lambda Topology issue -- more context in PRs for Python and JavaScript:
-
aws-observability/aws-otel-python-instrumentation#319
-
aws-observability/aws-otel-js-instrumentation#149

**Description of changes:**

- Apply fix for the Lambda Topology issue. The logic mimics the fix in
our other ADOT SDKs.
- Adding back AWS Resource support for Lambda.
-
aws-observability#907
  - We previously removed support due to the Lambda Topology issue

**Test plan:**

Set up two Lambda functions with Java runtimes and tested with custom
Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are
screenshots of the topology for various configurations.

**v1 Topology (lambdaA & lambdaB instrumented)**
<img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM"
src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7"
/>

**v2 Topology (lambdaA & lambdaB instrumented)**
<img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM"
src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5"
/>

We observe the following correct behaviors for topology above:
- Service entity node for `Invoke` call to downstream lambda.
- AWS Resource node for `GetFunction` call to downstream lambda.
- AWS Resource node for `ListBuckets` call to downstream s3.

**v1 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

**v2 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

We observe the following correct behaviors for topology above:
- Downstream lambda called with `Invoke` is correctly treated as
RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda
instrumentation patch behaves correctly.

**v1 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM"
src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5"
/>

**v2 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM"
src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2"
/>

**v1 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM"
src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02"
/>

**v2 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM"
src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2"
/>

We observe the following correct behaviors in the spans above:
- For `Invoke` calls, we see `aws.remote.service` and
`aws.remote.environment` correctly populated in the spans.
- For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource
attributes such as `aws.remote.resource.identifier` and
`aws.cloudformation.primary.identifier` correctly populated.


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
yiyuan-he added a commit to yiyuan-he/aws-otel-java-instrumentation that referenced this pull request May 28, 2025
@yiyuan-he
Fix: Lambda Topology Issue (aws-observability#1016)
0db4b31 
**Issue #, if available:**

Lambda Topology issue -- more context in PRs for Python and JavaScript:
-
aws-observability/aws-otel-python-instrumentation#319
-
aws-observability/aws-otel-js-instrumentation#149

**Description of changes:**

- Apply fix for the Lambda Topology issue. The logic mimics the fix in
our other ADOT SDKs.
- Adding back AWS Resource support for Lambda.
-
aws-observability#907
  - We previously removed support due to the Lambda Topology issue

**Test plan:**

Set up two Lambda functions with Java runtimes and tested with custom
Lambda layer with fix built-in. Tested both AWS SDK v1 and v2. Below are
screenshots of the topology for various configurations.

**v1 Topology (lambdaA & lambdaB instrumented)**
<img width="1311" alt="Screenshot 2025-02-07 at 11 48 51 AM"
src="https://github.com/user-attachments/assets/48234604-ae4b-49cd-926f-05cdd74038a7"
/>

**v2 Topology (lambdaA & lambdaB instrumented)**
<img width="1222" alt="Screenshot 2025-02-07 at 11 26 34 AM"
src="https://github.com/user-attachments/assets/cf7446f3-888f-4756-8ce0-e5ed1e97c9b5"
/>

We observe the following correct behaviors for topology above:
- Service entity node for `Invoke` call to downstream lambda.
- AWS Resource node for `GetFunction` call to downstream lambda.
- AWS Resource node for `ListBuckets` call to downstream s3.

**v1 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

**v2 Topology (lambdaB not instrumented)**
<img width="965" alt="Screenshot 2025-02-07 at 12 17 59 PM"
src="https://github.com/user-attachments/assets/67c361c0-4b8b-4d54-b1dd-0f21a9eee6ff"
/>

We observe the following correct behaviors for topology above:
- Downstream lambda called with `Invoke` is correctly treated as
RemoteService entity when not instrumented

Additionally, I generated the spans locally to verify the lambda
instrumentation patch behaves correctly.

**v1 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 23 PM"
src="https://github.com/user-attachments/assets/8d025453-4658-47c7-8c50-261be8b665f5"
/>

**v2 Invoke**
<img width="1281" alt="Screenshot 2025-02-06 at 10 11 49 PM"
src="https://github.com/user-attachments/assets/46b382d0-9475-4871-9773-ed78e609d4a2"
/>

**v1 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 08 53 PM"
src="https://github.com/user-attachments/assets/a59e2de6-d50c-47bf-b9d6-171c1ce7cc02"
/>

**v2 GetFunction**
<img width="1281" alt="Screenshot 2025-02-06 at 10 10 05 PM"
src="https://github.com/user-attachments/assets/bec349e0-92c7-4d80-b778-8fa29f3b1ab2"
/>

We observe the following correct behaviors in the spans above:
- For `Invoke` calls, we see `aws.remote.service` and
`aws.remote.environment` correctly populated in the spans.
- For non-`Invoke` calls (i.e. `GetFunction`), we see AWS Resource
attributes such as `aws.remote.resource.identifier` and
`aws.cloudformation.primary.identifier` correctly populated.


By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wangzlei wangzlei wangzlei approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yiyuan-he @wangzlei