-
Notifications
You must be signed in to change notification settings - Fork 341
Set an upper bound for the number of timeline events gathered #6089
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6089 +/- ##
==========================================
- Coverage 81.94% 81.28% -0.67%
==========================================
Files 473 468 -5
Lines 19560 19482 -78
==========================================
- Hits 16029 15835 -194
- Misses 3531 3647 +116 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Overall package sizeSelf size: 9.68 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.0.0 | 20.3 MB | 20.31 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.14 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
e18818a
to
16c0275
Compare
BenchmarksBenchmark execution time: 2025-07-15 08:17:48 Comparing candidate commit b1b5bd6 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 1273 metrics, 50 unstable metrics. |
6c0b312
to
672e321
Compare
672e321
to
23f4c49
Compare
What does this PR do?
Establishes an upper limit on the number of non-CPU timeline events that can be gathered within a single upload period.
Motivation
It's a bad idea for it to be unbounded.
Additional Notes
The upper limit is rather conservative and is heuristically derived from the CPU sampling rate, the upload period, and the number of libuv worker threads (and available parallelism.) E.g. if the CPU sampling rate is 10ms and we upload every 60s, that means at most 6000 CPU samples. We'd like to be able to observe oversubscription on the libuv thread pool. If there are 4 libuv threads, we'd like to be able to observe at least 6000*5 events (events executing on at least 4 threads + 1 waiting for detecting oversubscription). Finally, another 6000 events are allowed for GC events so that gives us
uploadPeriod / samplingPeriod * (threads + 2))
as the maximum number of events we wish to sample. With our defaults of 65s upload period, 99Hz sampling rate, and 4 libuv threads we'll collect at most 38613 events (when more are gathered, events get randomly discarded.) Because we already employ sampling I believe this upper bound is not very limiting in ordinary usage – its purpose is to prevent pathological memory usage runoff.Jira: PROF-12127