8354555: Add generic JFR events for TaskTerminator #24676

pengxiaolong · 2025-04-16T08:24:15Z

The purpose of the PR is to add generic JFR events for TaskTerminator to track the attempts and timings that GC threads have tried to terminate GC tasks.

Today only G1 emits JFR event with name Termination from G1ParEvacuateFollowersClosure, all other garbage collectors don't emit any JFR event for the termination attempt at all.

By adding this, it gives performance engineers the visibility to the termination attempts and termination time when GC threads trying to finish GC tasks, we could build tool to analyze the jfr events to determine if there is potential data structure issue in application code, e.g. very large LinkedList or LinkedBlockingQueue.

For the test, I have manually tested different GCs with Flight Recording enabled and verified the events:
G1:

jdk.GCPhaseParallel {
  startTime = 23:09:34.124 (2025-05-22)
  duration = 0.0108 ms
  gcId = 0
  gcWorkerId = 8
  name = "Termination"
  eventThread = "GC Thread#4" (osThreadId = 20483)
}

jdk.GCPhaseParallel {
  startTime = 23:09:34.124 (2025-05-22)
  duration = 0.0467 ms
  gcId = 0
  gcWorkerId = 2
  name = "Termination"
  eventThread = "GC Thread#2" (osThreadId = 21251)
}

jdk.GCPhaseParallel {
  startTime = 23:09:34.124 (2025-05-22)
  duration = 0.0474 ms
  gcId = 0
  gcWorkerId = 1
  name = "Termination"
  eventThread = "GC Thread#8" (osThreadId = 36359)
}
jdk.GCPhaseParallel {
  startTime = 23:09:41.925 (2025-05-22)
  duration = 0.000834 ms
  gcId = 14
  gcWorkerId = 7
  name = "Termination: Parallel Marking"
  eventThread = "GC Thread#1" (osThreadId = 21507)
}

jdk.GCPhaseParallel {
  startTime = 23:09:41.925 (2025-05-22)
  duration = 0.000166 ms
  gcId = 14
  gcWorkerId = 7
  name = "Termination: Parallel Marking"
  eventThread = "GC Thread#1" (osThreadId = 21507)
}

Shenandoah:

jdk.GCPhaseParallel {
  startTime = 23:39:58.890 (2025-05-22)
  duration = 0.0202 ms
  gcId = 0
  gcWorkerId = 0
  name = "Termination: Concurrent Mark"
  eventThread = "Shenandoah GC Threads#3" (osThreadId = 13827)
}

jdk.GCPhaseParallel {
  startTime = 23:39:58.890 (2025-05-22)
  duration = 0.0205 ms
  gcId = 0
  gcWorkerId = 1
  name = "Termination: Concurrent Mark"
  eventThread = "Shenandoah GC Threads#1" (osThreadId = 14339)
}

jdk.GCPhaseParallel {
  startTime = 23:39:58.890 (2025-05-22)
  duration = 0.0127 ms
  gcId = 0
  gcWorkerId = 5
  name = "Termination: Final Mark"
  eventThread = "Shenandoah GC Threads#1" (osThreadId = 14339)
}

jdk.GCPhaseParallel {
  startTime = 23:39:58.890 (2025-05-22)
  duration = 0.0377 ms
  gcId = 0
  gcWorkerId = 3
  name = "Termination: Final Mark"
  eventThread = "Shenandoah GC Threads#3" (osThreadId = 13827)
}

Parallel GC:

jdk.GCPhaseParallel {
  startTime = 23:31:25.009 (2025-05-22)
  duration = 0.0168 ms
  gcId = 0
  gcWorkerId = 4
  name = "Termination: Scavenge"
  eventThread = "GC Thread#1" (osThreadId = 14339)
}

jdk.GCPhaseParallel {
  startTime = 23:31:25.009 (2025-05-22)
  duration = 0.325 ms
  gcId = 0
  gcWorkerId = 5
  name = "Termination: Scavenge"
  eventThread = "GC Thread#5" (osThreadId = 21507)
}

jdk.GCPhaseParallel {
  startTime = 23:31:25.009 (2025-05-22)
  duration = 0.338 ms
  gcId = 0
  gcWorkerId = 8
  name = "Termination: Scavenge"
  eventThread = "GC Thread#7" (osThreadId = 33799)
}

Test

Test and verify the termination events in jfr for G1, Shenandoah and Parallel GC.
Tier 1
Tier 2
all tests in jdk/jfr

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8354555: Add generic JFR events for TaskTerminator (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/24676/head:pull/24676
$ git checkout pull/24676

Update a local copy of the PR:
$ git checkout pull/24676
$ git pull https://git.openjdk.org/jdk.git pull/24676/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 24676

View PR using the GUI difftool:
$ git pr show -t 24676

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/24676.diff

Using Webrev

Link to Webrev Comment

…tor::emit_termination_statistics to emit the event

bridgekeeper · 2025-04-16T08:24:56Z

👋 Welcome back xpeng! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-04-16T08:25:32Z

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

openjdk · 2025-04-16T08:26:07Z

@pengxiaolong The following labels will be automatically applied to this pull request:

hotspot
shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

shipilev

It looks compelling, but we need to fix a few issues:

src/hotspot/share/gc/g1/g1ConcurrentMark.cpp

src/hotspot/share/gc/g1/g1YoungCollector.cpp

src/hotspot/share/gc/parallel/psScavenge.cpp

src/hotspot/share/gc/shared/taskTerminator.cpp

src/hotspot/share/gc/shenandoah/shenandoahConcurrentMark.cpp

shipilev · 2025-05-06T15:09:52Z

src/hotspot/share/gc/shared/taskTerminator.cpp

+
+  ~TaskTerminationTracker() {
+    if (_terminator->_task_name != nullptr && _event.should_commit()) {
+      char* event_name = NEW_C_HEAP_ARRAY(char, strlen(TaskTerminator::termination_event_name_prefix) + strlen(_terminator->_task_name) + 1, mtGC);


Well, NEW_C_HEAP_ARRAY without a matching FREE_C_HEAP_ARRAY is a memory leak.

We do this to prepend Termination: , right? Can we just pass the constant strings to TaskTerminator, and avoid all this dynamicity? It would be a bit tedious, but I think you can do the compile-time concatenation with a few macros.

Thanks, I have replaced it with macro for the string concatenation here, now it doesn't use NEW_C_HEAP_ARRAY at all.

src/hotspot/share/gc/shared/taskTerminator.hpp

…RegionsBaseTask and G1STWRefProcProxyTask

pengxiaolong · 2025-05-13T22:24:46Z

src/hotspot/share/gc/g1/g1YoungCollector.cpp

-    _terminator(num_workers, _task_queues),
-    _pinned_regions_recorded(false)
-  { }
+    _terminator(num_workers, _task_queues, G1GCPhaseTimes::phase_name(G1GCPhaseTimes::Termination)),


G1EvacuateRegionsBaseTask emits Termination event from G1ParEvacuateFollowersClosure:: offer_termination(), I am trying to keep it same.

pengxiaolong · 2025-05-13T22:25:24Z

src/hotspot/share/gc/g1/g1YoungCollector.cpp

@@ -949,7 +946,7 @@ class G1STWRefProcProxyTask : public RefProcProxyTask {
    : RefProcProxyTask("G1STWRefProcProxyTask", max_workers),
      _g1h(g1h),
      _pss(pss),
-      _terminator(max_workers, &task_queues),
+      _terminator(max_workers, &task_queues, G1GCPhaseTimes::phase_name(G1GCPhaseTimes::Termination)),


RefProcProxyTask emits Termination event from G1ParEvacuateFollowersClosure:: offer_termination(), I am trying to keep it same as it is today.

mlbridge · 2025-05-14T06:00:30Z

Webrevs

openjdk · 2025-05-23T01:09:39Z

@pengxiaolong this pull request can not be integrated into master due to one or more merge conflicts. To resolve these merge conflicts and update this pull request you can run the following commands in the local repository for your personal fork:

git checkout JDK-8354555
git fetch https://git.openjdk.org/jdk.git master
git merge FETCH_HEAD
# resolve conflicts and follow the instructions given by git merge
git commit -m "Merge master"
git push

bridgekeeper · 2025-06-10T23:52:02Z

@pengxiaolong This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

pengxiaolong · 2025-06-24T23:59:08Z

/keepalive

openjdk · 2025-06-24T23:59:18Z

@pengxiaolong The pull request is being re-evaluated and the inactivity timeout has been reset.

bridgekeeper · 2025-07-23T02:18:41Z

@pengxiaolong This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply issue a /touch or /keepalive command to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

pengxiaolong added 9 commits April 15, 2025 09:57

Track termination timings and attempts in TaskTerminator

c848c3f

WorkerTask optionally returns TaskTerminator*

022a759

Add method to get WorkerDataStats based on data in WorkerDataArray

af63457

Add Jfr event EventGCTaskTerminationStatistics and method TaskTermina…

35c0849

…tor::emit_termination_statistics to emit the event

Fix typo

8d3fc3b

Fix build errors, namings

c57e2a1

Invoke emit_termination_statistics after finishing execution

cc26471

Enable new jfr event

32fdd14

Not emit event when count is 0

d3247cb

openjdk bot added hotspot hotspot-dev@openjdk.org shenandoah shenandoah-dev@openjdk.org labels Apr 16, 2025

pengxiaolong added 8 commits April 16, 2025 01:30

timings should be in ms

6316854

Fix build error

6b9cbe6

Emit EventGCPhaseParallel events to track task terminations

313e5c0

Fix wrong string concat

6b57aff

Revert changes in WorkerDataArray

ad9d156

Remove GCTaskTerminationStatistics event metadata

4cff56f

Clean up

fd25fda

Remove duplicate EventGCPhaseParallel for termination

3fc1207

shipilev reviewed May 6, 2025

View reviewed changes

pengxiaolong added 7 commits May 8, 2025 11:34

Address PR comments

34f75f4

Override termination event name for remark

e2a9446

termination_event_name must have prefix "Termination"

88998d0

Fix ident

b3374f5

Move TERMINATION_EVENT_NAME_PREFIX_ASSERT to taskTerminator.cpp

737f5de

Touch up

59d9da4

Add include "workerThread.hpp"

104ac1b

Emit exact same events for G1 as G1 is emitting today from G1Evacuate…

7b1412f

…RegionsBaseTask and G1STWRefProcProxyTask

pengxiaolong commented May 13, 2025

View reviewed changes

pengxiaolong marked this pull request as ready for review May 14, 2025 05:56

openjdk bot added the rfr Pull request is ready for review label May 14, 2025

Patch to fix the PR concerns

e362b7c

openjdk bot added the merge-conflict Pull request has merge conflict with target branch label May 23, 2025

Merge branch 'master' into JDK-8354555

5ab1c5f

openjdk bot removed the merge-conflict Pull request has merge conflict with target branch label May 23, 2025

pengxiaolong and others added 3 commits May 23, 2025 09:58

Fix jft test failure

fe67149

Merge branch 'openjdk:master' into JDK-8354555

17e3255

Merge branch 'openjdk:master' into JDK-8354555

8fb9a40

Merge branch 'openjdk:master' into JDK-8354555

c6ff707

pengxiaolong added 2 commits July 15, 2025 08:05

Merge branch 'openjdk:master' into JDK-8354555

7dd5ebf

Merge branch 'openjdk:master' into JDK-8354555

4d60e2e

8354555: Add generic JFR events for TaskTerminator #24676

Are you sure you want to change the base?

8354555: Add generic JFR events for TaskTerminator #24676

Uh oh!

Conversation

pengxiaolong commented Apr 16, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test

Progress

Issue

Reviewing

Uh oh!

bridgekeeper bot commented Apr 16, 2025

Uh oh!

openjdk bot commented Apr 16, 2025

Uh oh!

openjdk bot commented Apr 16, 2025

Uh oh!

shipilev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

shipilev May 6, 2025

Choose a reason for hiding this comment

Uh oh!

pengxiaolong May 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pengxiaolong May 13, 2025

Choose a reason for hiding this comment

Uh oh!

pengxiaolong May 13, 2025

Choose a reason for hiding this comment

Uh oh!

mlbridge bot commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

openjdk bot commented May 23, 2025

Uh oh!

bridgekeeper bot commented Jun 10, 2025

Uh oh!

pengxiaolong commented Jun 24, 2025

Uh oh!

openjdk bot commented Jun 24, 2025

Uh oh!

bridgekeeper bot commented Jul 23, 2025

Uh oh!

Uh oh!

pengxiaolong commented Apr 16, 2025 •

edited by openjdk bot

Loading

mlbridge bot commented May 14, 2025 •

edited

Loading