INV_MAX_COUNT increase / OpenDTU heap memory improvements

[mathieucarbou]

Is it safe and supported to change INV_MAX_COUNT to a bigger number in Configuration.h, and recompile the firmware ?

Also, why is the limit set to 10 and not more by default ?

Thanks!

[GizzGool]

Thank you in advance also for this modification.
If it's a CPU load problem, maybe do a scan in groups of 10 also ?

[tbnobody]

It will not work because also the amount of memory required for several json outputs has to be extended. For some outputs thats no more possible because they are on the memory limit. Therefor larger rewrites would be required.

[mathieucarbou]

So in order to go higher one would need to use Ahoy instead on a RPI for exemple ?

[mathieucarbou]

... or maybe not ? Ahoy seems to have a limit of 3 inverter ? https://github.com/lumapu/ahoy/search?q=MAX_INVERTER&type=code

[mathieucarbou]

@tbnobody : sorry to re-open this subject. My setup will now consist of 8 HMS-1000 and 6 HM-400. So I am running into this issue.

Did you specifically identified the places in the app where there are such memory bottlenecks ?

I am willing to take a look to reduce the heap memory usage.

[GizzGool]

I'm still interested too.
ESPs seem to have made great strides, so there should no longer be any memory issues?

[mathieucarbou]

Some additional comments:

• I am running with a firmware compiled with #define INV_MAX_COUNT 14 without issue to far (but I do not use prometheus)
• CONFIG_ASYNC_TCP_STACK_SIZE is not controlled: the default value for the async_tcp task is 16k, which is HUGE.
• -D CONFIG_ASYNC_TCP_STACK_SIZE=4096 I have tried moving it to 4k, which any app should be able to do, and it failed. Something is allocating a LOT of stack within the context of the async_tcp task, which is a bad smell. async_tcp task, like any arduino task, is run user-callbacks from async_tcp task which should be short and memory efficient, like any other arduino task like async_udp, etc. There is something wrong to look here. And this is not caused by ArduinoJson since ArduinoJson is allocating on heap (if latest version is used)
• Arduino Core 2 is still used (and core 2 is not supported anymore).
• There is a lot of stack or flash allocations that could be saved (i.e. INVERTER_CONFIG_T Inverter[INV_MAX_COUNT];). Dynamically allocating based on the real number of inverters would be better: here, someone with 5 invertersh has 50% of the allocated space unused
• CONFIG_ASYNC_TCP_EVENT_QUEUE_SIZE => could be removed
• CONFIG_ASYNC_TCP_QUEUE_SIZE => could be removed too and set back to teh default value of 64. Increasing it to 128 is also a bad smell (if 128 is required to fix a problem) and is also running for issues because chances are that heap memory will not have enough space for that.

These are some leads I found so far to improve.

[GizzGool]

Hello Mathieu,
Thank you very much for your feedback and tests.
After a few weeks of testing, is your feedback still OK with these modifications?
Is it possible to consider a firmware (perhaps specific and/or with a more powerful ESP?) with an OpenDTU supporting more than 10 inverters?
Thank you in advance and have a great summer.