fix: add config to process messages concurrently #2026

sairanjit · 2024-09-09T07:11:37Z

Add support to process the messages concurrently

cc: @Zzocker : Taking it further from #1276

@TimoGlastra From the earlier conversations how can we handle the queue from inbound transport?

Can we use the event emitter ?

changeset-bot · 2024-09-09T07:11:41Z

⚠️ No Changeset found

Latest commit: f9de361

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

TimoGlastra

Good updates! Thanks

packages/core/src/agent/AgentConfig.ts

packages/core/src/types.ts

TimoGlastra · 2024-09-13T08:45:59Z

packages/node/src/transport/WsInboundTransport.ts

+            filter((e) => e.payload.message.id === encryptedMessage.id),
+            filter((e) => e.payload.message.type === encryptedMessage.type),


an encrypted message does not have an id and type right?

Sorry, I missed this

But here there are no common properties I can map to. Can you suggest me a way to add a guard here or filter specific encrypted message?

packages/node/src/transport/HttpInboundTransport.ts

…ncurrently Signed-off-by: Pritam Singh <pkspritam16@gmail.com>

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

packages/node/src/transport/HttpInboundTransport.ts

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

TimoGlastra · 2024-09-27T18:09:11Z

packages/node/src/transport/HttpInboundTransport.ts

+        observable
+          .pipe(
+            filter((e) => e.type === AgentEventTypes.AgentMessageProcessed),
+            filter((e) => deepEquality(e.payload.encryptedMessage, encryptedMessage)),


I think we can just check for equality right (===)? AS the object will be passed around? Using deep object equality can be expensive

TimoGlastra · 2024-09-27T18:09:59Z

packages/core/src/agent/Dispatcher.ts

  }

-  public async dispatch(messageContext: InboundMessageContext): Promise<void> {
+  public async dispatch(messageContext: InboundMessageContext, encryptedMessage?: EncryptedMessage): Promise<void> {


What if we add encryptedMessage to the InboundMessageContext ( as it's related to the incoming message)

TimoGlastra · 2024-09-27T18:12:28Z

packages/node/src/transport/WsInboundTransport.ts

+            filter((e) => e.type === AgentEventTypes.AgentMessageProcessed),
+            filter((e) => deepEquality(e.payload.encryptedMessage, encryptedMessage)),
+            timeout({
+              first: 10000, // timeout after 10 seconds


I think it's less relevant here to await the processing? As we don't do anything afterwards

TimoGlastra · 2024-09-27T18:12:50Z

packages/node/src/transport/HttpInboundTransport.ts

+            filter((e) => e.type === AgentEventTypes.AgentMessageProcessed),
+            filter((e) => deepEquality(e.payload.encryptedMessage, encryptedMessage)),
+            timeout({
+              first: 10000, // timeout after 10 seconds


Let's add this as a param to the constructor of the HttpInboundTransport class?

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

TimoGlastra · 2024-10-10T13:53:52Z

@sairanjit can you resolve the conflicts? I think we're good to merge!

…t/concurrent-message-processing Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

sairanjit · 2024-10-10T14:28:18Z

@TimoGlastra Resolved conflicts now

TimoGlastra

Thanks so much for processing all the feedback!!

…on#2026) Signed-off-by: Pritam Singh <pkspritam16@gmail.com> Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com> Co-authored-by: Pritam Singh <pkspritam16@gmail.com> Signed-off-by: Martin Auer <martin.auer97@gmail.com>

…on#2026) Signed-off-by: Pritam Singh <pkspritam16@gmail.com> Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com> Co-authored-by: Pritam Singh <pkspritam16@gmail.com> Signed-off-by: Ariel Gentile <gentilester@gmail.com>

sairanjit marked this pull request as ready for review September 13, 2024 08:15

sairanjit force-pushed the feat/concurrent-message-processing branch from da1b2a1 to 3281bc5 Compare September 13, 2024 08:16

TimoGlastra reviewed Sep 13, 2024

View reviewed changes

Zzocker and others added 5 commits September 13, 2024 16:35

fix: add processMessagesConcurrently config for processing message co…

c8bf9f2

…ncurrently Signed-off-by: Pritam Singh <pkspritam16@gmail.com>

refactor: inbound transport to use AgentMessageReceivedEvent

0d1741a

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

refactor: inbound transport to use AgentMessageProcessedEvent

958310d

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

refactor: update name

a4e90c0

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

refactor: added timeout in message processing

9a37419

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

sairanjit force-pushed the feat/concurrent-message-processing branch from 05caec4 to 9a37419 Compare September 13, 2024 11:06

sairanjit added 2 commits September 13, 2024 16:37

chore: update lock file

806ef6a

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

Merge branch 'main' into feat/concurrent-message-processing

696afdf

TimoGlastra reviewed Sep 18, 2024

View reviewed changes

packages/node/src/transport/HttpInboundTransport.ts Show resolved Hide resolved

fix: AgentMessageProcessedEvent payload

93eaf72

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

TimoGlastra reviewed Sep 27, 2024

View reviewed changes

sairanjit added 2 commits September 29, 2024 15:31

fix: resolve comments

bd10210

Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

Merge branch 'main' into feat/concurrent-message-processing

fe7c5db

Merge branch 'main' of https://github.com/sairanjit/credo-ts into fea…

83cde0c

…t/concurrent-message-processing Signed-off-by: Sai Ranjit Tummalapalli <sairanjit.tummalapalli@ayanworks.com>

sairanjit requested a review from TimoGlastra October 10, 2024 14:28

Merge branch 'main' into feat/concurrent-message-processing

f9de361

TimoGlastra approved these changes Oct 11, 2024

View reviewed changes

TimoGlastra enabled auto-merge (squash) October 11, 2024 06:41

TimoGlastra merged commit 5a8f536 into openwallet-foundation:main Oct 11, 2024
14 checks passed

genaris mentioned this pull request Dec 5, 2024

fix: add concurrency config for processing message concurrently #1276

Closed

sairanjit mentioned this pull request Jan 8, 2025

Connection Reuse in edge wallets #2010

Closed

genaris mentioned this pull request Jul 11, 2025

fix(didcomm): support receivedAt in message receiver #2333

Merged

		filter((e) => e.payload.message.id === encryptedMessage.id),
		filter((e) => e.payload.message.type === encryptedMessage.type),

fix: add config to process messages concurrently #2026

fix: add config to process messages concurrently #2026

Uh oh!

Conversation

sairanjit commented Sep 9, 2024

Uh oh!

changeset-bot bot commented Sep 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

TimoGlastra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TimoGlastra Sep 13, 2024

Choose a reason for hiding this comment

Uh oh!

sairanjit Sep 13, 2024

Choose a reason for hiding this comment

Uh oh!

sairanjit Sep 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TimoGlastra Sep 27, 2024

Choose a reason for hiding this comment

Uh oh!

TimoGlastra Sep 27, 2024

Choose a reason for hiding this comment

Uh oh!

TimoGlastra Sep 27, 2024

Choose a reason for hiding this comment

Uh oh!

TimoGlastra Sep 27, 2024

Choose a reason for hiding this comment

Uh oh!

TimoGlastra commented Oct 10, 2024

Uh oh!

sairanjit commented Oct 10, 2024

Uh oh!

TimoGlastra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

changeset-bot bot commented Sep 9, 2024 •

edited

Loading

sairanjit Sep 13, 2024 •

edited

Loading