Use GC natively

[denniswittich]

Motivation

Rosys based systems lagged once every minute when the gc.collect() was called.

Implementation

Remove the mechanism that removd the native gc and instead collected once every minute.

Progress

• Discuss if the manual collection should be optional
• Remove unused code

[falkoschindler]

@denniswittich Some quick thoughts:

• Today we already merged PR Add a configuration for garbage_collection_mbyte_limit #300 allowing to configure the memory limit for garbage collection.
• I'm not sure we can simply enable the automatic garbage collection. Our experiments (a few years ago) should significant interruptions with this default setting. But I guess we need to discuss our options in more detail.
• Commenting out code in observe_memory_growth shouldn't make a difference as long as you don't call it in user code.

[evnchn]

By the way, if GC performance is important, consider gc.collect(2) then gc.freeze() after initialization and just before the server starts, so that those objects which are not collected at time are not considered ever in the GC process.

[NiklasNeugebauer]

@evnchn yes, good point. We might want to look into that since we do have a lot of long living objects after settings up most systems (and nicegui, rosys)

[evnchn]

Yes, especially since Python's GC is quadratic (at least more than linear) to the number of objects being processed, so the lower you can get the count the better things are.

[evnchn]

https://stackoverflow.com/a/33752634

Pypy, last time I checked, does not support running NiceGUI for some reason. We may consider to "make it work", since Pypy's GC does not "stop the world".

We may have to find a better place than a closed PR's comment area to discuss GC stuff for RoSys/NiceGUI

[evnchn]

PyPy and NiceGUI is not going to work without a fight...

And I'm fighting over at zauberzeug/nicegui#5159

💪🏻

[evnchn]

zauberzeug/nicegui#5163

[NiklasNeugebauer]

@evnchn
If pypy is incompatible with opencv then this will be the biggest road block.
I would not complain if we managed to drop opencv, but this will not happen in the short term.

[evnchn]

To clarify, opencv-python should be installable on PyPy but it's no good for the pipeline, taking a LONG time to install since there's no prebuilt wheel (at least that's what I think is the issue). After all it did install in some of my earlier testing but then there was some extra problem with numpy so I gave up.

Note with zauberzeug/nicegui#5163 my goal is to make NiceGUI run on PyPy, and I care about nothing else (and that's a good thing both to reduce the scope and to ensure me, who don't have physical access to the robots, don't propose some bad code that crashes them on the field)

Nevertheless, I would not necessarily promote dropping opencv in RoSys since OpenCV is good and has many examples online (meaning we can take inspiration when creating CV stuff)

[NiklasNeugebauer]

okay, I just saw that you removed it and expected it was incompatible. Thank you for clarifying.
Excited to see how this goes with NiceGUI then.