Add note about duplicate test execution with @Suite #5080

mpkorstanje · 2025-10-21T15:12:59Z

Users are surprised to see that their tests are executed twice[1]. This is the natural consequence of using a test engine to define a suite.

We can at least let them know in the documentation.

When I create a test suite the tests are runned twice, once independenly and in suite. [mvn test] #4327

I hereby agree to the terms of the JUnit Contributor License Agreement.

Definition of Done

There are no TODOs left in the code
Method preconditions are checked and documented in the method's Javadoc
Coding conventions (e.g. for logging) have been followed
Change is covered by automated tests including corner cases, errors, and exception handling
Public API has Javadoc and @API annotations
Change is documented in the User Guide and Release Notes

Users are surprised to see that their tests are executed twice[1]. This is the natural consequence of using a test engine to define a suite. We can at least let them know in the documentation. 1. #4327

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc

marcphilipp · 2025-10-22T06:54:59Z

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc

+===== Duplicate test execution
+
+Tests in a suite are executed in addition to the tests executed by every other test
+engine. This can result in the same tests being executed twice.


Should we add sth. like the following?

Moreover, different suites may contain the same tests, potentially with different setups.

Maybe it is time to add a drawing or two. I'll come back to that. Something along the lines of:

https://github.com/cucumber/cucumber-jvm/tree/main/cucumber-junit-platform-engine#suites-with-different-configurations

@mpkorstanje Do you have time to add the drawing for 6.0.1 or should we move this PR to 6.1.0-M1?

I've moved it to 6.1.0-M1. We don't have to rush this one. It's not a bug.

Should we add sth. like the following?

Done.

…uite-engine-documentation

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

documentation/src/plantuml/launcher-api-diagram.puml

documentation/documentation.gradle.kts

marcphilipp

I think the diagrams are very helpful! 👍

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc

documentation/src/plantuml/launcher-api-diagram.puml

documentation/src/docs/asciidoc/user-guide/advanced-topics/launcher-api.adoc

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc

junit-platform-suite-api/src/main/java/org/junit/platform/suite/api/Suite.java

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Co-authored-by: Marc Philipp <mail@marcphilipp.de>

…ncher-api.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

…e/api/Suite.java Co-authored-by: Marc Philipp <mail@marcphilipp.de>

junit-platform-suite-api/src/main/java/org/junit/platform/suite/api/Suite.java

Users are surprised to see that their tests are executed twice[1]. This is the natural consequence of using a test engine to define a suite. We can at least let them know in the documentation. 1. #4327 Co-authored-by: Marc Philipp <mail@marcphilipp.de> (cherry picked from commit c08e421)

mpkorstanje · 2025-10-31T12:26:10Z

Cherry picked to releases/6.0.x in 866c01a

mpkorstanje added 2 commits October 21, 2025 17:12

Add note duplicate test execution with @suite

16ff13b

Users are surprised to see that their tests are executed twice[1]. This is the natural consequence of using a test engine to define a suite. We can at least let them know in the documentation. 1. #4327

Update pointers and commands in readme

c0a8776

mpkorstanje force-pushed the rien/junit-platform-suite-engine-documentation branch from 076b9c3 to c0a8776 Compare October 21, 2025 15:41

mpkorstanje changed the title ~~Add note duplicate test execution with @Suite~~ Add note about duplicate test execution with @Suite Oct 21, 2025

marcphilipp reviewed Oct 21, 2025

View reviewed changes

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc Outdated Show resolved Hide resolved

mpkorstanje added 3 commits October 21, 2025 18:54

Extract and update note to independent section

8546b97

Update Suite annotation to match

6f2dacd

Update Suite annotation to match

f74202e

marcphilipp approved these changes Oct 22, 2025

View reviewed changes

documentation/src/docs/asciidoc/user-guide/advanced-topics/junit-platform-suite-engine.adoc Outdated Show resolved Hide resolved

marcphilipp reviewed Oct 22, 2025

View reviewed changes

marcphilipp added this to the 6.0.1 milestone Oct 24, 2025

marcphilipp assigned mpkorstanje Oct 28, 2025

mpkorstanje modified the milestones: 6.0.1, 6.1.0-M1 Oct 29, 2025

mpkorstanje and others added 5 commits October 30, 2025 13:21

Merge remote-tracking branch 'origin/main' into rien/junit-platform-s…

36365a3

…uite-engine-documentation

Update documentation/src/docs/asciidoc/user-guide/advanced-topics/jun…

890a5b7

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Draft diagrams

c3a88d0

Rework, rewrite, revise

fcd8f7e

Remove ellipsis, spell Jupiter correctly

5e02295

mpkorstanje commented Oct 30, 2025

View reviewed changes

documentation/src/plantuml/launcher-api-diagram.puml Show resolved Hide resolved

mpkorstanje commented Oct 30, 2025

View reviewed changes

documentation/documentation.gradle.kts Show resolved Hide resolved

Rework, rewrite, revise

b822014

mpkorstanje requested a review from marcphilipp October 30, 2025 18:37

Rename diagrams to match header ids

4df8e81

marcphilipp approved these changes Oct 31, 2025

View reviewed changes

mpkorstanje and others added 5 commits October 31, 2025 13:05

Update documentation/src/docs/asciidoc/user-guide/advanced-topics/jun…

aa2d097

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Update documentation/src/docs/asciidoc/user-guide/advanced-topics/jun…

bef798d

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Update documentation/src/plantuml/launcher-api-diagram.puml

ed4a741

Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Update documentation/src/docs/asciidoc/user-guide/advanced-topics/lau…

05fb4fd

…ncher-api.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Update documentation/src/docs/asciidoc/user-guide/advanced-topics/jun…

c132c99

…it-platform-suite-engine.adoc Co-authored-by: Marc Philipp <mail@marcphilipp.de>

Update junit-platform-suite-api/src/main/java/org/junit/platform/suit…

970d710

…e/api/Suite.java Co-authored-by: Marc Philipp <mail@marcphilipp.de>

mpkorstanje commented Oct 31, 2025

View reviewed changes

junit-platform-suite-api/src/main/java/org/junit/platform/suite/api/Suite.java Outdated Show resolved Hide resolved

Merge paragraphs

da41307

mpkorstanje merged commit c08e421 into main Oct 31, 2025
14 checks passed

mpkorstanje deleted the rien/junit-platform-suite-engine-documentation branch October 31, 2025 12:12

mpkorstanje modified the milestones: 6.1.0-M1, 6.0.1 Oct 31, 2025

Uh oh!

Add note about duplicate test execution with @Suite #5080

Add note about duplicate test execution with @Suite #5080

Conversation

mpkorstanje commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Definition of Done

Uh oh!

Uh oh!

Uh oh!

marcphilipp Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp Oct 24, 2025

Choose a reason for hiding this comment

Uh oh!

marcphilipp Oct 29, 2025

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mpkorstanje Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

marcphilipp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mpkorstanje commented Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mpkorstanje commented Oct 21, 2025 •

edited

Loading

mpkorstanje Oct 29, 2025 •

edited

Loading

mpkorstanje Oct 30, 2025 •

edited

Loading