Why do I need so large mbedtls heap ?!

[joelguittet]

Hello community!

I previously discussed issues to have multiple TLS sockets opened at the same time, that finally conclude the mbedtls heap size was not enough, and increasing CONFIG_MBEDTLS_HEAP_SIZE solved my issues so my application was properly working.

From 60000 bytes I increased to 100000 bytes !!! Yes you properly read that, almost 100kB to get my application working, while I was doing 2/3 TLS sockets at the same time (only). If not increasing this I was getting ENOMEM error while opening the sockets.

That sound crazy to reserve this amount of memory for this, so I wonder why ? Have you got throw similar issues ? If yes can you share about your experience so that we can discuss of the reasons/solutions/etc around this topic.

Joel

[hraftery]

Not quite so extreme, but I was shocked that even increasing to 40000 was not enough for me. Early in the comms process the library allocates an in buffer and an out buffer, each 16713 (MBEDTLS_SSL_IN_BUFFER_LEN) bytes long. Even that seems excess in the embedded world, particularly when there are already tx and rx network buffers in the stack, like CONFIG_NET_BUF_DATA_SIZE.

But 40000 wasn't enough, and I noticed there's a lot of dynamic allocation to make copies of the certificates in the chain. Since neither the chain nor the certificate have a clear max, that's easy another 5*4000 bytes that might be required. I guess it's not too much of a stretch to see why you need 100000 with multiple sockets.

I had to go to 48000 because I stopped getting alloc failures. I am thankful that every alloc call seems to be checked for failure (unlike the wpa_supplicant driver... grrrr!).

So I'm at the point where, even on a purpose built nrf5340 with 512kB of RAM and a dedicated WiFi co-processor, I'm desperately running out of RAM to run a single TLS session over WiFi.

[joelguittet]

Hello @hraftery
So you do similar observation, I m happy to not be alone ! But not sure how we can do better personally.
Joel

[brettNel]

Any headway here? I too ran into this and your findings (thank you for posting them!!) helped me get my dual TLS connections up (CONFIG_MBEDTLS_HEAP_SIZE=100000). This seems insane to me.

[hraftery]

I ended up scrapping gains from elsewhere.

https://devzone.nordicsemi.com/f/nordic-q-a/112158/getting-config_heap_mem_pool_size-under-control-minimising

Ultimately I managed to divorce myself from NCS. I don’t see it improving any time soon. It’ll be npm level of bloated dependency hell before long. Not a place I want to see the embedded world go.

[joelguittet]

@brettNel I have not challenged this topic since a long time now, still using the 100000 configuration.
One point to start should be to have a clear understanding of the memory usage. If you know some tool to track memory usage across time, this could help I think (so we identify room for improvements).
For a comparison note that I faced the same problem with ThreadX RTOS where I had to increase the memory allocated to the TLS usage up to a similar amount.
So TLS seems to need this amount of memory, that's the point. And when you design an application, choosing the right MCU is a key point. This is not just dealing with the number of GPIOs or the presence of the peripherals, here RAM size is an important factor.

[bjarki-andreasen]

I did some digging as well when working with the nRF7002 WiFi drivers using the script found in zephyr/scripts/footprint/size_report and I identified two wild memory hogs, one being LIBC having its own 40K heap, alongside the 140K zephyr heap required for WiFi etc. which largely comes from seemingly needing enourmous buffers for sockets if NM_WPA_SUPPLICANT is used.

The reason NM_WPA_SUPPLICANT requires huge socket buffers seems to be that it is based on the BSD socket abstraction, which requires COPYING, hence, many big buffers...