[viosock] After installing Viosocklib.dll, the app fails to create new socket

[xuehuihui]

Steps to reproduce

Reproduce this problem through a simple code. The code flow is as follows

1. WSAStartup(MAKEWORD(2, 2), &wsaData) initiates use of the Winsock DLL by a process.
2. socket(family, SOCK_STREAM, 0) create a socket
3. cmd command line to execute install viosocklib.dll or update viosock.inf
   • viosocklib-test.exe /d
   • viosocklib-test.exe /i
4. socket(family, SOCK_STREAM, 0) Create a new socket

Test Results

Step 4: Creating a new socket will fail with error code 10022

Issue Analysis

code stack

socket(family, SOCK_STREAM, 0)--->VIOSockSocket--->g_UpcallTable.lpWPUModifyIFSHandle

g_UpcallTable.lpWPUModifyIFSHandle failed: 10022

solve

Add the following code after the step 3, the second socket will be created successfully.

WSACleanup();
errCode = WSAStartup(MAKEWORD(2, 2), &wsaData);
if (errCode != ERROR_SUCCESS || HIBYTE(wsaData.wVersion != 2) || LOBYTE(wsaData.wVersion) != 2) {
	printf("WSAStartup Failed or ! Error Code : %d\r\n", GetLastError());
	return -1;
}

client.zip

[xuehuihui]

I think this is expected behavior and I'm just seeking to understand the reason behind it. Additionally, I'm interested in knowing if there's a way to install Viosocklib.dl without immediate effect, such as only taking effect after a Windows restart.

[MartinDrab]

Hello,

what are you trying to accomplish with the step 3)? I do not think WinSock is ready for that. During its second registration, viosocklib.dll can get a different CatalogIdEntry (e.g. somebody else register his/her provider before viosocklib-test /i gets executed) which is then passed to lpWPUModifyIFSHandle. On provider unregistration, WinSock apparently cleans up some of its data structure, thus, the second socket creation does not work.

Similarly, if you do not re-register the provider, the second socket creation attempt fails as well but with different error (WSAEAFNOSUPPORT). Although this provider could, in theory, work even then (it mostly relays requests to the viosock.sys driver), WinSock is not happy with such a scenario.

Martin

[xuehuihui]

Thanks for your reply. I can probably understand

what are you trying to accomplish with the step 3)?
==》
I mainly want to verify what will happen when upgrading the viosock driver when using vsock

[MartinDrab]

if there's a way to install Viosocklib.dl without immediate effect, such as only taking effect after a Windows restart.

I do not think there is a documented way of doing that. You can try to do the registration yourself by directly writing necessary entries into the registry. I am not sure whether WinSock2 checks the registry whenever asked about providers though.