`--filter FilterWhichDoesNotMatchAnything` should exit with non-zero exit code

[staabm]

Q	A
PHPUnit version	a62124f
PHP version	8.3.23
Installation Method	Composer / PHAR

How to reproduce

➜  phpunit git:(main) php phpunit --filter FilterWhichDoesNotMatchAnything          
PHPUnit 12.3-ga62124f38 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.23
Configuration: /Users/m.staab/Documents/GitHub/phpunit/phpunit.xml

No tests executed!
➜  phpunit git:(main) echo $?            
0

➜  phpunit git:(main) php -v                                              
PHP 8.3.23 (cli) (built: Jul  1 2025 16:52:12) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.3.23, Copyright (c) Zend Technologies

Expected behavior

I think in case we do not run any tests and had a --filter defined, PHPUnit should return a non-zero exit code

[staabm]

I think the same should be true for --testsuite and --group based filters:

➜  phpunit git:(main) php phpunit --group test-abc
PHPUnit 12.3-ga62124f38 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.23
Configuration: /Users/m.staab/Documents/GitHub/phpunit/phpunit.xml

No tests executed!
➜  phpunit git:(main) echo $?
0
➜  phpunit git:(main) php phpunit --testsuite test-abc
PHPUnit 12.3-ga62124f38 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.23
Configuration: /Users/m.staab/Documents/GitHub/phpunit/phpunit.xml

No tests executed!
➜  phpunit git:(main) echo $?                         
0

[kubawerlos]

What about running without any parameters and still getting "No tests executed!" (because of no tests)?

What about a simple relation, "No tests executed!" -> "a non-zero exit code"?

[staabm]

this might work, too.

I tried to talk about my actual use-cases. but it might work for a more general case.

not sure about possible BC impact though.

[joteejotee]

I'd like to work on this issue if nobody else is working on it.

[staabm]

We have no signal from Sebastian yet whether a fix would be accepted (he is on vacation atm).

But if you want to work on it - go ahead.

[joteejotee]

That makes sense—since we haven’t heard Sebastian’s opinion yet, I’ll hold off for now. Thanks for letting me know!

[stof]

There is already a --fail-on-empty-test-suite config setting (the XML config file also supports it). Isn't it solving that need already ?

[staabm]

Isn't it solving that need already ?

thanks, wasn't aware of --fail-on-empty-test-suite.

I think the behaviour of --fail-on-empty-test-suite should be the default for --filter (or maybe even in general).

[stof]

@staabm just set failOnEmptyTestSuite="true" in your XML config file (or failOnAllIssues="true").

Right now, none of the failOn* setting defaults to true, so this is consistent.