Help with evaluation: A few queries

[Roh-codeur]

hi team,
I am evaluating this lib for my organization. I am wondering if this supports the below use cases. upon reading docs, I can see some of them are.
Publisher sends messages to a few topics
Multiple subscriber starts up
the subscribers should receive existing messages and then subsequent updates
Compaction/conflation
In order for above to work, messages with same id should be conflated/compacted. Subscribers should only ever receive the latest message for an id
Server side filtering
I want to be able to subscribe to a topic, but, with a filter. so, say, a topic has messages, I shoudl be able to subscribe with filter for id = xx. I should then receive state of world and subsequent updates with that filter
thanks!

[MauriceVanVeen]

Hi 👋 , indeed support your use case.

You can use JetStream to publish into a stream and have multiple consumers to receive both existing and new messages. You'll get that behavior if you set the DeliverPolicy to All to get all messages.

Filters can be used to do server side filtering based on the subject, the consumer will then only get messages that match. With the deliver policy of all mentioned above, you'd get all messages matching the filter, and any subsequent messages.

[Roh-codeur]

perfect, have been reading the docs and am super impressed. can you please point me to some relevant parts of docs for my use cases please? I will then try to put together a POC. Also, if you have any code examples which help me put my POC together please

thanks!

[Roh-codeur]

Also, is there a way to conflate or compact messages on the server. so, consumers only receive the latest message for an id, can't find this in the docs. thanks

[MauriceVanVeen]

DeliverPolicy and FilterSubject(s) can be found here:
https://docs.nats.io/nats-concepts/jetstream/consumers#configuration

A code example can be found here:
https://natsbyexample.com/examples/jetstream/pull-consumer/go

Only getting one/latest message for a subject/id can be done in two ways, depending on what you need to achieve.
Can either use DeliverPolicy: DeliverLastPerSubject to initially get the last message per subject and subsequent new updates after. Or if you only ever want to store one message per subject/id, you can use MaxMsgsPerSubject: 1:
https://docs.nats.io/nats-concepts/jetstream/streams#configuration

[Roh-codeur]

got it. I was just reading this: https://docs.nats.io/nats-concepts/subjects

I guess what you are suggesting is:

1. create a topic like: symbol.mktdata
2. now while publishing, publish data as symbol.mktdata.A, symbol.mktdata.B etc.
3. now, I can use filtering by topic symbol.mktdata.A.
4. and configure topic as DeliverPolicy: DeliverLastPerSubject

I will try this out. I am using python on a Mac. I hope all of this is supported

thanks again