Skip to content

Fix the reconnection logic for parallel writes with HA Producer. #335

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 22, 2024

Conversation

hiimjako
Copy link
Collaborator

If there are parallel calls to Send or BatchSend in concurrent routines during reconnection, only one of these sends will unlock and send the message, while the others will remain in a locked state. This is because currently there isn't a broadcast notification with reconnectionSignal.
To fix this issue the reconnectionSignal channel has been changed to a sync.Cond.

@Gsantomaggio
Copy link
Member

Great find! Thanks @hiimjako

May I ask to change the example by adding a number for concurrent producers?, like:

   const concurrentProducers = xxx

Then you should test the example with these parameters by restarting the node.

I use:

declare -a arr=$(docker ps -q)

for image in $(docker ps -q); do
   echo Stopping ..$image
   docker restart $image
   echo Waiting 60s ..$image 
   sleep 40
   echo Waiting 20s ..$image
   sleep 20
   echo Waiting 10s ..$image
   sleep 10
   echo Waiting 10s ..$image
   sleep 10
   echo Waiting 10s ..$image
   sleep 10
done 

And I leave it running for some time. The result numbers should be coherent.
thank you

@hiimjako
Copy link
Collaborator Author

May I ask to change the example by adding a number for concurrent producers?

Done!
In the main branch the example returns more or less half of the messages, as expected, while in this branch all 20M messages.

@Gsantomaggio Gsantomaggio merged commit cb4e0d2 into rabbitmq:main Jul 22, 2024
2 checks passed
@Gsantomaggio
Copy link
Member

Thank you

@hiimjako hiimjako added the bug Something isn't working label Jul 29, 2024
@hiimjako hiimjako self-assigned this Jul 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants