Test infrastructure relies on end-of-life software

[jgonggrijp]

The tests currently rely on Karma and PhantomJS. I love PhantomJS, but it is sadly very dead. The last update was in 2018 and the author has archived the repository. It does not run out of the box on my current PC. There might be a way to kick it back into action, but I'm not even going to try it, because it is so outdated and there are modern options that render it obsolete.

Karma was recently deprecated. It still works and will receive security patches for a while, but eventually, that road is a dead end, too.

The most urgent problem, PhantomJS, is relatively easy to work around. In local development, the developer can open the tests in a browser of choice instead. In CI, the tests already run in Sauce Labs, so there is no need to run them in PhantomJS as well. I plan to implement these changes soon.

Karma is more effort to replace. I'm leaning towards Web Test Runner, which can run headless tests in modern Chromium, Firefox and WebKit via Playwright and which also has a dedicated launcher for Sauce Labs. There is third-party support for QUnit as well. If that third-party support fails, however, we might need to rewrite all tests to a different test framework, which is obviously a lot of work.

If we end up needing to port the tests to a different framework, I propose that it be Mocha. Mocha is old, trusted, actively maintained and widely supported. In particular, it enjoys built-in support from Web Test Runner. It also makes it possible to use power-assert. However, let's not go here as long as we can make QUnit work without much effort.

I welcome pull requests that replace Karma by Web Test Runner.

Tangentially related: #4271.

[paulfalgout]

👍 for mocha. Works for so many runners, and is generally understood by most.