Skip to content

Set an upper bound for the number of timeline events gathered #6089

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

szegedi
Copy link
Contributor

@szegedi szegedi commented Jul 14, 2025

What does this PR do?

Establishes an upper limit on the number of non-CPU timeline events that can be gathered within a single upload period.

Motivation

It's a bad idea for it to be unbounded.

Additional Notes

The upper limit is rather conservative and is heuristically derived from the CPU sampling rate, the upload period, and the number of libuv worker threads (and available parallelism.) E.g. if the CPU sampling rate is 10ms and we upload every 60s, that means at most 6000 CPU samples. We'd like to be able to observe oversubscription on the libuv thread pool. If there are 4 libuv threads, we'd like to be able to observe at least 6000*5 events (events executing on at least 4 threads + 1 waiting for detecting oversubscription). Finally, another 6000 events are allowed for GC events so that gives us uploadPeriod / samplingPeriod * (threads + 2)) as the maximum number of events we wish to sample. With our defaults of 65s upload period, 99Hz sampling rate, and 4 libuv threads we'll collect at most 38613 events (when more are gathered, events get randomly discarded.) Because we already employ sampling I believe this upper bound is not very limiting in ordinary usage – its purpose is to prevent pathological memory usage runoff.

Jira: PROF-12127

Copy link

codecov bot commented Jul 14, 2025

Codecov Report

Attention: Patch coverage is 63.15789% with 21 lines in your changes missing coverage. Please review.

Project coverage is 81.28%. Comparing base (195f0ce) to head (b1b5bd6).

Files with missing lines Patch % Lines
...ackages/dd-trace/src/profiling/profilers/events.js 43.24% 21 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6089      +/-   ##
==========================================
- Coverage   81.94%   81.28%   -0.67%     
==========================================
  Files         473      468       -5     
  Lines       19560    19482      -78     
==========================================
- Hits        16029    15835     -194     
- Misses       3531     3647     +116     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

github-actions bot commented Jul 14, 2025

Overall package size

Self size: 9.68 MB
Deduped: 109.26 MB
No deduping: 109.65 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.7.0 | 35.02 MB | 35.02 MB | | @datadog/native-appsec | 10.0.0 | 20.3 MB | 20.31 MB | | @datadog/native-iast-taint-tracking | 4.0.0 | 11.72 MB | 11.73 MB | | @datadog/pprof | 5.9.0 | 9.77 MB | 10.14 MB | | @opentelemetry/core | 1.30.1 | 908.66 kB | 7.16 MB | | protobufjs | 7.5.3 | 2.95 MB | 5.6 MB | | @datadog/wasm-js-rewriter | 4.0.1 | 2.85 MB | 3.58 MB | | @datadog/native-metrics | 3.1.1 | 1.02 MB | 1.43 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | jsonpath-plus | 10.3.0 | 617.18 kB | 1.08 MB | | import-in-the-middle | 1.14.2 | 122.36 kB | 850.93 kB | | lru-cache | 10.4.3 | 804.3 kB | 804.3 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.1 | 109.9 kB | 109.9 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 7.0.5 | 63.38 kB | 63.38 kB | | istanbul-lib-coverage | 3.2.2 | 34.37 kB | 34.37 kB | | rfdc | 1.4.1 | 27.15 kB | 27.15 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | dc-polyfill | 0.1.9 | 25.11 kB | 25.11 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | shell-quote | 1.8.3 | 23.74 kB | 23.74 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | semifies | 1.0.0 | 15.84 kB | 15.84 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | mutexify | 1.4.0 | 5.71 kB | 8.74 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.4 | 3.96 kB | 3.96 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@szegedi szegedi force-pushed the szegedi/timeline-event-max branch 2 times, most recently from e18818a to 16c0275 Compare July 14, 2025 14:59
@pr-commenter
Copy link

pr-commenter bot commented Jul 14, 2025

Benchmarks

Benchmark execution time: 2025-07-15 08:17:48

Comparing candidate commit b1b5bd6 in PR branch szegedi/timeline-event-max with baseline commit 195f0ce in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 1273 metrics, 50 unstable metrics.

@szegedi szegedi force-pushed the szegedi/timeline-event-max branch 2 times, most recently from 6c0b312 to 672e321 Compare July 14, 2025 15:21
@szegedi szegedi force-pushed the szegedi/timeline-event-max branch from 672e321 to 23f4c49 Compare July 14, 2025 15:38
@szegedi szegedi marked this pull request as ready for review July 14, 2025 15:49
@szegedi szegedi requested a review from a team as a code owner July 14, 2025 15:49
@szegedi szegedi enabled auto-merge (squash) July 14, 2025 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant