junit as ToolProvider and misbehaving tests

[sormuras]

Forked discussion from here: #4469 (comment)

Note that the launcher implements ToolProvider interface in order to allow usages without spawning a new VM.

Now assuming my build tool relies on the ToolProvider interface to launch JUnit in my own VM, how am I expected to deal with the user's shutdown hooks? I can no longer use the workaround to pass the user's classpath in -cp, because that's my own classpath, so what's the suggested approach?

What if a test calls directly (or indirectly in production code) System.exit(123);, and a SecurityManager (JEP 411, JEP 486) is not installed or no longer supported?

[joffrey-bion]

Thanks a lot @sormuras for taking the time to discuss this and follow up.

Indeed it looks like everything that relate to the lifecycle of the JVM process itself (System.exit, shutdown hooks, etc.) might be a problem in the test environment if we want to run multiple sessions in the same JVM.

In the JEP 486 appendix, an example is shown about how to use an agent to intercept calls to this kind of Java APIs. I wonder if that could be a way to go for JUnit.

I don't believe it's completely far-fetched to imagine a world where the test session is sandboxed to some extent, so that such API calls can be intercepted and even maybe reacted to by test authors. This is why I was suggesting maybe intercepting addShutdownHook and running the hooks at the end of the session rather than when the JVM actually shuts down. Something similar could also be done with System.exit.