Link on lower-memory microcontroller

[jacobschloss]

I'm trying to use cyclone on a microcontroller with Zephyr 4.2 with 320kB of sram, of which about 250kB is free before using cyclone. It seems to build ok after applying a few small patches for header files, the networking patch in cyclone's tree, and removing a few things like cyclone's version of recvmsg which zephyr seems to provide now (CONFIG_POSIX_NETWORKING, CONFIG_XSI_REALTIME).

The first call to dds_create_participant makes me overflow sram by 86kB, so it seems like cyclone is pulling about 300kB-400kB of compile time data into the noinit section.

Are there any tunables to reduce static ram use?
Any thoughts on the minimum ram requirements for a microcontroller to use cyclone?

... did some looking around ...

It looks like the bulk of it is from the threads stacks with defaults sizes defined at thread.c lines 239 / 243.
CYCLONEDDS_THREAD_COUNT
CYCLONEDDS_THREAD_STACK_SIZE

Any thoughts on a reasonable number of threads for a low message rate (eg ~100-500Hz, small <100B messages)? Eg, setting CYCLONEDDS_THREAD_COUNT to 2?

I'll do some testing with CYCLONEDDS_THREAD_COUNT=2 and CYCLONEDDS_THREAD_STACK_SIZE=16384. This is able to fit easily.

[eboasson]

Hi @jacobschloss, Cyclone currently always creates 5 threads in its default configuration:

• tev for timed events, sending discovery data and sending retransmits;
• recv for reading from the network sockets and delivering data to the reader history caches / queueing data for asynchronous processing by a dq.* thread;
• dq.user, for asynchronously delivering data into reader history caches;
• dq.builtin for processing discovery data;
• gc for deferred deleting of data (garbage collection) and lease expiration checking.

There can be more threads: latency budget > 0 gives rise to a thread, you can have multiple receive threads, there can be a thread for monitoring whether the regular threads keep making progress — but those are threads you can easily avoid.

So reducing CYCLONEDDS_THREAD_COUNT to 5 (or perhaps 6, considering there is also the application thread?) seems feasible without further modifications.

If you're willing to hack a bit in Cyclone, it should be possible to reduce the number of threads a bit further. For example, dq.user does not really have work to do in the default configuration because the default configuration delivers all data synchronously (on the recv thread). IINM that means it only does something when writers disappear, but if no data at all is processed by that thread, then that step can be skipped. See

cyclonedds/src/core/ddsi/src/ddsi_proxy_endpoint.c

Line 484 in d8c1bb6

struct ddsi_gcreq *gcreq = ddsi_gcreq_new (pwr->e.gv->gcreq_queue, gc_delete_proxy_writer_dqueue);

and

cyclonedds/src/core/ddsi/src/ddsi_proxy_endpoint.c

Line 479 in d8c1bb6

ddsi_dqueue_enqueue_callback (dqueue, gc_delete_proxy_writer_dqueue_bubble_cb, gcreq);

that's where it deletes the proxy writer by first enqueuing a bubble that the dq.user thread processes, which then continues the cleanup on the gc thread thanks to

cyclonedds/src/core/ddsi/src/ddsi_proxy_endpoint.c

Line 470 in d8c1bb6

ddsi_gcreq_requeue (gcreq, gc_delete_proxy_writer);

Short-circuiting that probably suffices to make dq.user never do anything, and if so, then that thread no longer needs to be created at all.

I also don't think that dq.builtin is strictly necessary. It makes sense to off-load discovery work to another thread so that the data flow continues while all the matching is happening, but if threads are costly, then doing that on the recv thread might make sense.

cyclonedds/src/core/ddsi/src/ddsi_receive.c

Line 2283 in d8c1bb6

int ddsi_user_dqueue_handler (const struct ddsi_rsample_info *sampleinfo, const struct ddsi_rdata *fragchain, const ddsi_guid_t *rdguid, UNUSED_ARG (void *qarg))

is where data coming out of dq.user is handled. If you look around a bit, you see it either enqueues data or calls deliver_user_data_synchronously, which tweaks things a bit so that ultimately the same function can do the actual work, no matter which path got taken.

For discovery data,

cyclonedds/src/core/ddsi/src/ddsi_discovery.c

Line 176 in d8c1bb6

int ddsi_builtins_dqueue_handler (const struct ddsi_rsample_info *sampleinfo, const struct ddsi_rdata *fragchain, UNUSED_ARG (const ddsi_guid_t *rdguid), UNUSED_ARG (void *qarg))

is where the data is handled. It seems to me that deliver_(user_)data_synchronously could check the writer GUID to decide whether the data is "user" data or "builtin" data and call the appropriate function. (It is "user" data if (guid[15] & 0xc0)==0.) There might a be a snag or two, but it'd be worth a try.

Getting rid of tev and gc would be tricky ... with some effort, I imagine tev could be merged with recv, but gc needs to be able to do some work in the background.

If the above really works out, then you could reduce the number of threads by quite a bit. And if you do it, then please consider doing it as (compile-time) options and open a PR with the changes :)

[jacobschloss]

Thanks, this was helpful. I'll do some more hacking around!