build_score_cache tests exceed 6h time limits

[j-i-l]

This happens when installing via the new installation procedure with pak, both on Fedora and on MacOS

Since the runner just hits the 6h time limit there is no informative output on where the tests take up a lot of time.