What would be a good way to batch-save consumed Jetstream messages? #472
Unanswered
robertmircea
asked this question in
Q&A
Replies: 1 comment
-
Good question. asking around, @ripienaar suggested another one of his world class solutions (thanks): So, you can create a consumer with AckPolicy=All, then use fetch and once the messages are persisted in the other storage, acking the last message would ack all the messages just fetched. the only caveat is the consumer can only have a single client for this to work correctly. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am using a durable consumer to retrieve up to 1000 messages at a time from a Jetstream stream. I am looking for an efficient way to batch save these messages from the NATS server into a different persistent storage. Specifically, I want to accumulate up to 'z' messages in a batch list and then persist the entire batch to storage in a single operation. Additionally, if the batch list does not fill up within 'x' seconds of the last save operation, it should be saved regardless of its size. Messages from NATS should only be acknowledged after the batch has been successfully saved. Example code would be welcomed.
Beta Was this translation helpful? Give feedback.
All reactions