Why a 64 MB message size limit?

[Marko19907]

Hi,
I noticed that NATS has a 64 MB message size limit, while similar solutions sometimes allow up to 1 GB or have no fixed limit. Could someone explain the rationale behind this cap? Also, if I fork the project and increase the limit myself to, say, 512 MB, what potential issues should I be aware of, are there concerns beyond just performance?

Thanks for any insights!

[neilalexander]

With messages that big, you will probably run out of memory very quickly, and if you didn't, you wouldn't be able to buffer full messages before triggering a slow consumer scenario on a subscriber.

[Marko19907]

Thanks for the explanation @neilalexander !

For our use case, we only have a single message (around 300 MB) at the app’s start, and we're comfortable with the increased memory footprint in that instance. Would it be reasonable to allow users to override the default limit, perhaps via a configuration option in the server's config file, and issue a warning (similar to RabbitMQ) when a high value is set? This could let advanced users decide if the trade-off is acceptable while also keeping the defaults safe for most deployments.

I’d love to hear your thoughts on this, are there other concerns that might make such configurability problematic?

Thanks!

[Rlopez]

So instead of us being the final user of NATS, we are the final user of a NATS utility that implements this sort of thing. That absolutely is -potentially- a great way to solve the issue... not c# support, though...

[Jarema]

@Rlopez actually c# client had it before the extensions system in the core client, so there is c# support: https://github.com/nats-io/nats.net/blob/main/src/NATS.Client.Core/NatsConnection.RequestReply.cs#L87

[Rlopez]

Nice! I'll keep that in mind when we get to chunking! :D
Thanks! :)

[Marko19907]

Just bumping this feature request. For our use case, handling a single ~300 MB message at startup would benefit from a configurable override (with an appropriate warning) for the message size limit. Is this something that might be considered in the future?