-
Notifications
You must be signed in to change notification settings - Fork 315
Context API beforeFinish Migration
#9422
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
* Creates inferred proxy spans for API Gateway calls via presence of http headers --------- Co-authored-by: Zarir Hamza <zarir.hamza@datadoghq.com>
Avoid duplicate expensive context extraction Avoid subclassing tracing span for serverless but used serverless context element instead to store / track inferred span while keep tracing feature untouched Improved propagator to not create / capture inferred span context element on invalid data Rework context element to hold the inferred spans and its captured data Release captured data as soon as they span start (never read after this point so reclaiming memory) Refactor context element and propagator into the right package, not context component (product / feature agnostic) Refactor unit tests
|
🎯 Code Coverage 🔗 Commit SHA: 5f20aee | Docs | Was this helpful? Give us feedback! |
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 51 metrics, 14 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1047936
Total [baseline] (8.783 s) : 0, 8782812
Agent [candidate] (1.04 s) : 0, 1039808
Total [candidate] (8.73 s) : 0, 8729891
section iast
Agent [baseline] (1.161 s) : 0, 1160591
Total [baseline] (9.364 s) : 0, 9363592
Agent [candidate] (1.16 s) : 0, 1159982
Total [candidate] (9.366 s) : 0, 9365876
gantt
title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.494 ms) : 0, 1494
crashtracking [candidate] (1.491 ms) : 0, 1491
BytebuddyAgent [baseline] (713.669 ms) : 0, 713669
BytebuddyAgent [candidate] (709.239 ms) : 0, 709239
GlobalTracer [baseline] (249.588 ms) : 0, 249588
GlobalTracer [candidate] (246.886 ms) : 0, 246886
AppSec [baseline] (33.135 ms) : 0, 33135
AppSec [candidate] (32.6 ms) : 0, 32600
Debugger [baseline] (6.662 ms) : 0, 6662
Debugger [candidate] (6.492 ms) : 0, 6492
Remote Config [baseline] (696.27 µs) : 0, 696
Remote Config [candidate] (686.034 µs) : 0, 686
Telemetry [baseline] (14.686 ms) : 0, 14686
Telemetry [candidate] (13.962 ms) : 0, 13962
Flare Poller [baseline] (6.54 ms) : 0, 6540
Flare Poller [candidate] (7.1 ms) : 0, 7100
section iast
crashtracking [baseline] (1.469 ms) : 0, 1469
crashtracking [candidate] (1.452 ms) : 0, 1452
BytebuddyAgent [baseline] (823.053 ms) : 0, 823053
BytebuddyAgent [candidate] (822.404 ms) : 0, 822404
GlobalTracer [baseline] (233.961 ms) : 0, 233961
GlobalTracer [candidate] (234.342 ms) : 0, 234342
AppSec [baseline] (30.677 ms) : 0, 30677
AppSec [candidate] (28.684 ms) : 0, 28684
Debugger [baseline] (6.085 ms) : 0, 6085
Debugger [candidate] (6.08 ms) : 0, 6080
Remote Config [baseline] (611.408 µs) : 0, 611
Remote Config [candidate] (598.001 µs) : 0, 598
Telemetry [baseline] (8.468 ms) : 0, 8468
Telemetry [candidate] (8.33 ms) : 0, 8330
Flare Poller [baseline] (4.13 ms) : 0, 4130
Flare Poller [candidate] (4.209 ms) : 0, 4209
IAST [baseline] (30.788 ms) : 0, 30788
IAST [candidate] (32.536 ms) : 0, 32536
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.043 s) : 0, 1042732
Total [baseline] (10.85 s) : 0, 10850326
Agent [candidate] (1.031 s) : 0, 1030752
Total [candidate] (10.869 s) : 0, 10869416
section appsec
Agent [baseline] (1.203 s) : 0, 1202839
Total [baseline] (10.991 s) : 0, 10991193
Agent [candidate] (1.203 s) : 0, 1203155
Total [candidate] (10.907 s) : 0, 10907393
section iast
Agent [baseline] (1.169 s) : 0, 1169356
Total [baseline] (11.179 s) : 0, 11179053
Agent [candidate] (1.159 s) : 0, 1159386
Total [candidate] (11.117 s) : 0, 11116799
section profiling
Agent [baseline] (1.18 s) : 0, 1180423
Total [baseline] (10.931 s) : 0, 10931115
Agent [candidate] (1.178 s) : 0, 1178020
Total [candidate] (10.871 s) : 0, 10871356
gantt
title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.484 ms) : 0, 1484
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (710.907 ms) : 0, 710907
BytebuddyAgent [candidate] (702.747 ms) : 0, 702747
GlobalTracer [baseline] (247.538 ms) : 0, 247538
GlobalTracer [candidate] (245.391 ms) : 0, 245391
AppSec [baseline] (32.82 ms) : 0, 32820
AppSec [candidate] (32.126 ms) : 0, 32126
Debugger [baseline] (6.531 ms) : 0, 6531
Debugger [candidate] (6.36 ms) : 0, 6360
Remote Config [baseline] (694.812 µs) : 0, 695
Remote Config [candidate] (684.084 µs) : 0, 684
Telemetry [baseline] (14.671 ms) : 0, 14671
Telemetry [candidate] (14.552 ms) : 0, 14552
Flare Poller [baseline] (6.632 ms) : 0, 6632
Flare Poller [candidate] (6.321 ms) : 0, 6321
section appsec
crashtracking [baseline] (1.462 ms) : 0, 1462
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (725.879 ms) : 0, 725879
BytebuddyAgent [candidate] (726.928 ms) : 0, 726928
GlobalTracer [baseline] (236.4 ms) : 0, 236400
GlobalTracer [candidate] (236.069 ms) : 0, 236069
AppSec [baseline] (174.193 ms) : 0, 174193
AppSec [candidate] (173.826 ms) : 0, 173826
Debugger [baseline] (5.864 ms) : 0, 5864
Debugger [candidate] (5.873 ms) : 0, 5873
Remote Config [baseline] (629.615 µs) : 0, 630
Remote Config [candidate] (648.271 µs) : 0, 648
Telemetry [baseline] (8.374 ms) : 0, 8374
Telemetry [candidate] (8.363 ms) : 0, 8363
Flare Poller [baseline] (3.853 ms) : 0, 3853
Flare Poller [candidate] (3.845 ms) : 0, 3845
IAST [baseline] (25.081 ms) : 0, 25081
IAST [candidate] (25.037 ms) : 0, 25037
section iast
crashtracking [baseline] (1.486 ms) : 0, 1486
crashtracking [candidate] (1.462 ms) : 0, 1462
BytebuddyAgent [baseline] (830.186 ms) : 0, 830186
BytebuddyAgent [candidate] (822.816 ms) : 0, 822816
GlobalTracer [baseline] (234.768 ms) : 0, 234768
GlobalTracer [candidate] (233.369 ms) : 0, 233369
AppSec [baseline] (30.105 ms) : 0, 30105
AppSec [candidate] (29.592 ms) : 0, 29592
Debugger [baseline] (6.162 ms) : 0, 6162
Debugger [candidate] (6.127 ms) : 0, 6127
Remote Config [baseline] (599.639 µs) : 0, 600
Remote Config [candidate] (593.44 µs) : 0, 593
Telemetry [baseline] (8.46 ms) : 0, 8460
Telemetry [candidate] (8.443 ms) : 0, 8443
Flare Poller [baseline] (4.161 ms) : 0, 4161
Flare Poller [candidate] (4.122 ms) : 0, 4122
IAST [baseline] (31.836 ms) : 0, 31836
IAST [candidate] (31.539 ms) : 0, 31539
section profiling
ProfilingAgent [baseline] (110.095 ms) : 0, 110095
ProfilingAgent [candidate] (110.2 ms) : 0, 110200
crashtracking [baseline] (1.483 ms) : 0, 1483
crashtracking [candidate] (1.454 ms) : 0, 1454
BytebuddyAgent [baseline] (731.138 ms) : 0, 731138
BytebuddyAgent [candidate] (729.662 ms) : 0, 729662
GlobalTracer [baseline] (221.725 ms) : 0, 221725
GlobalTracer [candidate] (220.755 ms) : 0, 220755
AppSec [baseline] (33.227 ms) : 0, 33227
AppSec [candidate] (33.308 ms) : 0, 33308
Debugger [baseline] (9.185 ms) : 0, 9185
Debugger [candidate] (9.861 ms) : 0, 9861
Remote Config [baseline] (717.634 µs) : 0, 718
Remote Config [candidate] (2.191 ms) : 0, 2191
Telemetry [baseline] (12.984 ms) : 0, 12984
Telemetry [candidate] (10.742 ms) : 0, 10742
Flare Poller [baseline] (4.18 ms) : 0, 4180
Flare Poller [candidate] (4.155 ms) : 0, 4155
Profiling [baseline] (110.751 ms) : 0, 110751
Profiling [candidate] (110.858 ms) : 0, 110858
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 1 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section baseline
no_agent (4.398 ms) : 4348, 4447
. : milestone, 4398,
iast (9.796 ms) : 9631, 9960
. : milestone, 9796,
iast_FULL (13.863 ms) : 13591, 14135
. : milestone, 13863,
iast_GLOBAL (11.004 ms) : 10793, 11214
. : milestone, 11004,
profiling (8.873 ms) : 8727, 9020
. : milestone, 8873,
tracing (7.786 ms) : 7675, 7897
. : milestone, 7786,
section candidate
no_agent (4.256 ms) : 4201, 4310
. : milestone, 4256,
iast (9.354 ms) : 9197, 9511
. : milestone, 9354,
iast_FULL (14.313 ms) : 14026, 14599
. : milestone, 14313,
iast_GLOBAL (10.561 ms) : 10374, 10748
. : milestone, 10561,
profiling (8.839 ms) : 8690, 8989
. : milestone, 8839,
tracing (7.7 ms) : 7588, 7813
. : milestone, 7700,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section baseline
no_agent (37.456 ms) : 37145, 37767
. : milestone, 37456,
appsec (47.461 ms) : 47046, 47875
. : milestone, 47461,
code_origins (43.4 ms) : 43022, 43778
. : milestone, 43400,
iast (45.229 ms) : 44833, 45625
. : milestone, 45229,
profiling (49.802 ms) : 49324, 50279
. : milestone, 49802,
tracing (43.535 ms) : 43151, 43918
. : milestone, 43535,
section candidate
no_agent (37.748 ms) : 37453, 38043
. : milestone, 37748,
appsec (47.154 ms) : 46734, 47574
. : milestone, 47154,
code_origins (46.065 ms) : 45658, 46472
. : milestone, 46065,
iast (44.541 ms) : 44150, 44932
. : milestone, 44541,
profiling (49.861 ms) : 49403, 50320
. : milestone, 49861,
tracing (42.935 ms) : 42567, 43302
. : milestone, 42935,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section baseline
no_agent (1.475 ms) : 1463, 1486
. : milestone, 1475,
appsec (3.726 ms) : 3506, 3946
. : milestone, 3726,
iast (2.213 ms) : 2149, 2278
. : milestone, 2213,
iast_GLOBAL (2.26 ms) : 2196, 2325
. : milestone, 2260,
profiling (2.082 ms) : 2028, 2136
. : milestone, 2082,
tracing (2.037 ms) : 1987, 2087
. : milestone, 2037,
section candidate
no_agent (1.48 ms) : 1468, 1491
. : milestone, 1480,
appsec (3.709 ms) : 3490, 3927
. : milestone, 3709,
iast (2.212 ms) : 2148, 2276
. : milestone, 2212,
iast_GLOBAL (2.255 ms) : 2191, 2319
. : milestone, 2255,
profiling (2.505 ms) : 2333, 2677
. : milestone, 2505,
tracing (2.039 ms) : 1989, 2089
. : milestone, 2039,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~5f20aeea23, baseline=1.55.0-SNAPSHOT~f69e04325c
dateFormat X
axisFormat %s
section baseline
no_agent (15.555 s) : 15555000, 15555000
. : milestone, 15555000,
appsec (15.086 s) : 15086000, 15086000
. : milestone, 15086000,
iast (18.333 s) : 18333000, 18333000
. : milestone, 18333000,
iast_GLOBAL (17.941 s) : 17941000, 17941000
. : milestone, 17941000,
profiling (15.465 s) : 15465000, 15465000
. : milestone, 15465000,
tracing (15.139 s) : 15139000, 15139000
. : milestone, 15139000,
section candidate
no_agent (15.288 s) : 15288000, 15288000
. : milestone, 15288000,
appsec (14.767 s) : 14767000, 14767000
. : milestone, 14767000,
iast (18.792 s) : 18792000, 18792000
. : milestone, 18792000,
iast_GLOBAL (18.029 s) : 18029000, 18029000
. : milestone, 18029000,
profiling (15.297 s) : 15297000, 15297000
. : milestone, 15297000,
tracing (15.206 s) : 15206000, 15206000
. : milestone, 15206000,
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only had a quick review so far but it looks promising 👍
We need to #9384 review an merged to prevent conflicts. Can you target this branch instead?
Can you also remove what's already part of #9388? It will get better to review if we keep this PR related to the beforeFinish only. I will target mine onto yours then.
| public AgentSpan beforeFinish(final AgentSpan span) { | ||
| return span; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❔ question: Can this one be removed then? It will prevent other decorator to override it and never being called.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cannot, WebSocketDecorator and its related instrumentation still call it. Needs to be a separate mini-migration
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that the only one blocker? Because the WebSocketDecorator is calling the empty BaseDecorator.beforeFinish() and has no override... So technically, the call is doing nothing :/
So we can move it from BaseDecorator to WebSocketDecorator and migrate it in a second pass.
@amarziali What's the effort to migrate the WebSocket instrumentation to context rather than span?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another place where this is called is within the Mule Instrumentation as well via the MuleDecorator in mule-4.
Moved the empty call to WebSocketDecorator and MuleDecorator thus allowing me to remove the function from BaseDecorator.
I assume the actual span -> context functionality will be taken care of later for both WebSockets and Mule.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I take it back, I'm not sure how the tests aren't catching it but the ClientDecorator also uses beforeFinish(span) via the BaseDecorator. This seems to indicate that we need to invest time into looking through all decorators and confirming their reliance on beforeFinish. To save time, for now we can focus just on HttpServerDecorator related instrumentation
...kka-http-10.0/src/main/java/datadog/trace/instrumentation/akkahttp/DatadogWrapperHelper.java
Outdated
Show resolved
Hide resolved
...kka-http-10.0/src/main/java/datadog/trace/instrumentation/akkahttp/DatadogWrapperHelper.java
Outdated
Show resolved
Hide resolved
...0/src/main/java/datadog/trace/instrumentation/grizzly/GrizzlyHttpHandlerInstrumentation.java
Show resolved
Hide resolved
...http-2.3.20/src/main/java/datadog/trace/instrumentation/grizzlyhttp232/GrizzlyDecorator.java
Show resolved
Hide resolved
...dertow-2.0/src/main/java/datadog/trace/instrumentation/undertow/ExchangeEndSpanListener.java
Outdated
Show resolved
Hide resolved
...bootstrap/src/main/java/datadog/trace/bootstrap/instrumentation/decorator/BaseDecorator.java
Outdated
Show resolved
Hide resolved
| public AgentSpan beforeFinish(final AgentSpan span) { | ||
| return span; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that the only one blocker? Because the WebSocketDecorator is calling the empty BaseDecorator.beforeFinish() and has no override... So technically, the call is doing nothing :/
So we can move it from BaseDecorator to WebSocketDecorator and migrate it in a second pass.
@amarziali What's the effort to migrate the WebSocket instrumentation to context rather than span?
...entation/cxf-2.1/src/main/java/datadog/trace/instrumentation/cxf/InvokerInstrumentation.java
Outdated
Show resolved
Hide resolved
...ain/java/datadog/trace/instrumentation/grizzlyhttp232/DefaultFilterChainInstrumentation.java
Outdated
Show resolved
Hide resolved
...zly-http-2.3.20/src/main/java/datadog/trace/instrumentation/grizzlyhttp232/FilterAdvice.java
Outdated
Show resolved
Hide resolved
...zly-http-2.3.20/src/main/java/datadog/trace/instrumentation/grizzlyhttp232/FilterAdvice.java
Show resolved
Hide resolved
...http-2.3.20/src/main/java/datadog/trace/instrumentation/grizzlyhttp232/GrizzlyDecorator.java
Outdated
Show resolved
Hide resolved
...t-1.1/src/main/java/datadog/trace/instrumentation/jaxrs/v1/JaxRsClientV1Instrumentation.java
Outdated
Show resolved
Hide resolved
| @@ -53,42 +55,58 @@ public void methodAdvice(MethodTransformer transformer) { | |||
|
|
|||
| static class HandleAdvice { | |||
| @Advice.OnMethodEnter(suppress = Throwable.class) | |||
| static AgentScope onEnter( | |||
| static ContextScope onEnter( | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
📝 notes: Note for myself. Time boxed the review to 1h and paused here for Today
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I miss baggage about Jetty instrumentation but mixing main context and dispatch span feels different than the original behavior. Did I miss something?
...ver-7.0/src/main/java/datadog/trace/instrumentation/jetty70/ServerHandleInstrumentation.java
Outdated
Show resolved
Hide resolved
...0/src/main/java/datadog/trace/instrumentation/jetty9/JettyCommitResponseInstrumentation.java
Outdated
Show resolved
Hide resolved
...rver-9.0/src/main/java/datadog/trace/instrumentation/jetty9/ServerHandleInstrumentation.java
Outdated
Show resolved
Hide resolved
...0/src/main/java_jetty10/datadog/trace/instrumentation/jetty10/JettyCommitResponseHelper.java
Outdated
Show resolved
Hide resolved
...rver-9.0/src/main/java_jetty10/datadog/trace/instrumentation/jetty10/ServerHandleAdvice.java
Outdated
Show resolved
Hide resolved
...src/main/java_jetty904/datadog/trace/instrumentation/jetty904/JettyCommitResponseHelper.java
Outdated
Show resolved
Hide resolved
...0/src/main/java_jetty93/datadog/trace/instrumentation/jetty93/JettyCommitResponseHelper.java
Outdated
Show resolved
Hide resolved
...c/main/java_jetty9421/datadog/trace/instrumentation/jetty9421/JettyCommitResponseHelper.java
Outdated
Show resolved
Hide resolved
|
System Tests are fixed here - DataDog/system-tests#5430 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👋 I didn't get the chance to fully review all files but I do have a concern.
Whenever scope.context() is called, we may need to make sure that the scope used is a ContextScope. If it is invoked on an AgentScope (like it is in many files now), we will get an AgentSpan back, which will lead to the wrong implementation of beforeFinish being called and the InferredProxySpan not actually getting finished.
Curious to what @PerfectSlayer and @mcculls think about this.
...tp-10.0/src/main/java/datadog/trace/instrumentation/akkahttp/DatadogAsyncHandlerWrapper.java
Outdated
Show resolved
Hide resolved
...instrumentation/axway-api/src/main/java/datadog/trace/instrumentation/axway/StateAdvice.java
Outdated
Show resolved
Hide resolved
...umentation/axway-api/src/main/java/datadog/trace/instrumentation/axway/HTTPPluginAdvice.java
Outdated
Show resolved
Hide resolved
| if (throwable != null) { | ||
| DECORATE.onError(span, throwable); | ||
| DECORATE.beforeFinish(span); | ||
| DECORATE.beforeFinish(scope.context()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same thing here, may need to migrate to ContextScope.
What Does This Do
Finishes context API migration in regards to
beforeFinishusing context instead of spanRefactors everything to use context instead of span across requests
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any usefull labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]