[CI]: `mypy-primer(0)` timeout due to long checking of `colour`

[randolf-scholz]

In several recent PRs the mypy-primer (0) can take close to, or over an hour causing timeout due to long checking on the colour package.

• Try to fix problems caused by in-place modification of typevar defaults #19382: 55 min, over 50 min spent on colour
• Fix Enum.value inference for Enums with @cached methods #19374: 55 min, over 50 min spent on colour
• perf: add __slots__ to SubtypeContext #19397: 58 min, over 56 min spent on colour
• Draft: Solve intermediate variable bug #19399: timeout, over 56 min spent on colour

[sterliakov]

Primer can be rebalanced, there are weights assigned to every project proportional to the check time. colour might need its own shard.

[randolf-scholz]

But these have to be defined over in https://github.com/hauntsaninja/mypy_primer, right? So I should move this issue there?

[randolf-scholz]

I tried running mypy locally on a fresh clone of colour, and it doesn't take nearly as long. It does take >600MB, so maybe the runner goes into swap if it runs mypy on multiple repos in parallel?

Command exited with non-zero status 1
	Command being timed: "mypy colour/ --no-incremental"
	User time (seconds): 83.03
	System time (seconds): 0.45
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 1:23.51
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 622748
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 201966
	Voluntary context switches: 11
	Involuntary context switches: 908
	Swaps: 0
	File system inputs: 51825
	File system outputs: 199294
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 1

[sterliakov]

How beefy is your PC? I tried running mypy on colour before, and it did take 30+ mins.

Yes, the weights should be adjusted in that repo, you can try opening a PR there or pinging @hauntsaninja for comments:) AFAIC I brought this up before and we agreed that rebalancing is the way to go, optionally along with adding a 6th shard for colour alone.

[randolf-scholz]

I do have a relatively recent build with an intel 265K and DDR5. Though I did run with mypy 1.16.1 rather than the most recent master, so maybe there is a regression.

[sterliakov]

Hm. Did you run compiled or interpreted mypy? A month or so ago I got ~50 mins using interpreted mypy on a laptop with Intel Core i3 with 32G DDR4 RAM.

If I remember correctly, primer runs interpreted mypy too.

[randolf-scholz]

So I did a rerun (using --no-incremental and deleting .mypy_cache)

• With mypy 1.18.0+dev.4a427e9f1ac8a007d5c9bea946cfeb9e163cfa89 (compiled: no) it takes roughly 8 min 30 sec
• With mypy 1.18.0+dev.4a427e9f1ac8a007d5c9bea946cfeb9e163cfa89 (compiled: yes) roughly 1 min 20 sec

The former makes the 50+ min runtime on a weak machine seem more plausible.