Data not distributed equally between two nodes

[mrajmohan1999]

I left a comment earlier that got removed so reposting this. I am facing an issue where I have two servers configured, but only one of them is being hit and storing data. One has 0 hits and the other has ~95% hit rate. This issue recently started about a couple months ago and we haven't made any changes on how the nodes are configured on our end. Both servers are alive and reachable so it isn't the issue of one server being down. The only error we have found is in the Read() method:

stack trace:
at Enyim.Caching.Memcached.PooledSocket.Read(Byte[] buffer, Int32 offset, Int32 count)
at Enyim.Caching.Memcached.Protocol.Binary.BinaryResponse.Read(PooledSocket socket)
at Enyim.Caching.Memcached.Protocol.Binary.BinarySingleItemOperation.ReadResponse(PooledSocket socket)

We tried using the text protocol as well as the binary and faced the same issue.

[adison88]

@cnblogs-dudu could you please help check on this issue?

[cnblogs-dudu]

Please telnet memcached servers and check curr_connections from stats.

[mrajmohan1999]

One node has a very small number of curr_connections (2) while the other has a lot more (over 20). We would expect both of them to have a similar number ideally

[mrajmoh1]

@cnblogs-dudu do you have any more suggestions on how we can troubleshoot?