Working group proposal: Test classloading

[holly-cummins]

Goal

At the moment, Quarkus tests are invoked using one classloader, and then executed in a different classloader. This mostly works well, but means some use cases don't work: extensions cannot manipulate test classes in the same way that they do normal application classes. For example, anything run via a JUnit @TestTemplate test case will see the un-transformed class.

It also means we have extra user-facing complexity, such as the QuarkusTest*Callbacks](https://quarkus.io/guides/getting-started-testing#enrichment-via-quarkustestcallback):

While it is possible to use JUnit Jupiter callback interfaces like BeforeEachCallback, you might run into classloading issues because Quarkus has to run tests in a custom classloader which JUnit is not aware of.

A final benefit is a reduction in the internal complexity of our code. Hopping between classloaders during test execution takes a lot of work, and adds a lot of code! It also is brittle in places. For example, because the hop between classloaders relies on serialization in some cases, it's becoming harder to do as the JVM tightens up security restrictions. We used to rely on xstream, but that stopped working in Java 17. In #40601, @dmlloyd moved us to use the JBoss Serializer, which works better, but might still be affected by future restrictions on class access.

The goal of this working group is to allow test classes to fully participate in the 'quarkification' of classes. The mechanism for this is probably just to load the test classes with the classloader we intend to run them with, so that JUnit sees the 'correct' version of the class.

Progress tracking

The following GitHub issues cover the problems we're seeing:

• Quarkus CDI-managed beans not available in Pact state callbacks #22611
• Adding @QuarkusTest support for PactVerifyProvider pact-foundation/pact-jvm#1506
• Ability to use UniAsserter with @ParameterizedTest #24590
• Templated test contexts do not see transformed test classes (classloader/ordering issues in tests) #27821
• Kotlin junit ParameterizedTest with functions as argument, not working anymore (Kotlin 2.0) #42000
• Regression in 3.13.0.CR1: Lambda expression from custom serializable interface used as @QuarkusTest method parameter fails with ClassNotFoundException #42006
• Mockito fails to mock non-public inner class in continuous testing due to classloading issues #38987
• Deserialization Errors Caused by Avro Classes being in the Wrong ClassLoader for Tests #36812
• MockitoExtension is ignored during continuous testing #27383
• @QuarkusTest classloader problems when executing XPathFactory.newInstance() #8451
• @RunOnVertxContext incompatible with @ParameterizedTest #39752

I'm less sure about the issues at the bottom of the list, they're "maybes".

The first big set of changes is tracked by PR #34681, although there will probably be pre-work needed to add more tests, and post-work needed to tidy up code paths, docs, and start deprecating QuarkusTest*Callbacks if possible.

Realistic Aim

This is a smaller rewrite of classloading than we've done in the past, so with proper focus, it should be achievable in a reasonable time frame.

End in Sight

This work is done when tests are loaded with the classloader they're run with, and obvious bits of vestigial code and docs have been cleaned up.

[dmlloyd]

+1 from me, and I'd be interested in joining this group. I could see this effort possibly morphing into a more general review of class loading in the future as well.

[geoand]

+1 from me as well

[maxandersen]

I can't commit to join/be regular but +1 on it being important and very curious to hear/follow progress and help testing if need arises!