Certificate failing to validate on Windows, Openssl, MSQUIC

[cjgatto]

I have a client key and cert for my msquic client, and a server key and cert for my msquic server. Certificates are trusted by Digicert CA.

I've compiled msquic with openssl. The msquic server runs on a Linux system, the client can run on Linux or Windows.

On Linux, the client and server authenticate just fine.

On Windows, the connections fails. The Linux server logs msquic status code QUIC_STATUS_BAD_CERTIFICATE, the Windows client logs a code of 0x8041012a but that doesn't seem to correlate to a status in msquic_winuser.h. If I set QUIC_CREDENTIAL_FLAG_NO_CERTIFICATE_VALIDATION on the Windows client, the connection works.

When running my certs through openssl.exe x509 -in sample.cert -text -noout, the results look normal. The fact that these certs work fine on Linux-to-Linux systems suggests there isn't an issue with the certs themselves.

The only thing I can think of is the openssl built and linked by msquic isn't picking up any necessary Window's certificate chains. Is there something I can try to diagnose this further? Using CMake on Windows, I built msquic with -DQUIC_TLS=openssl.

[nibanks]

Windows client logs a code of 0x8041012a but that doesn't seem to correlate to a status in msquic_winuser.h

0x8041012a comes from a combination of:

#define QUIC_TLS_ALERT_HRESULT_PREFIX   _HRESULT_TYPEDEF_(0x80410100L)

#define QUIC_STATUS_TLS_ALERT(Alert)        (QUIC_TLS_ALERT_HRESULT_PREFIX | (0xff & Alert))

#define QUIC_STATUS_BAD_CERTIFICATE         QUIC_STATUS_TLS_ALERT(42)   // Bad Certificate

Because 0x2a equals 42, so QUIC_STATUS_TLS_ALERT(42) equals 0x8041012a.

Now, as to why Windows doesn't like the certificate, the Windows logs more information to ETW about the specific errors. My guess is that for some reason, Windows doesn't trust the root of the cert by default. Try grabbing the logs to find out more.

[shashenli]

seems like this is a useful way to solve it

[shashenli]

but if OpenSSL does not load certs installed on Windows where will OpenSSL load certs?

[cjgatto]

You'll have to find that in the OpenSSL docs, X509_LOOKUP_load_file for example loads a cert file to an in-memory cert store

[shashenli]

So if i run client on linux it will ask OS to load CA cert?but if i run client on windows it will not

[shashenli]

if i catch the certificate received event in a callback what should i do to perform validation with the received cert and my CA. would u pls show me how u do on code? ur help will be appreciated