Can dakota.sandia.gov supply full SSL cert chain for curl, python, wget, other simple user-agents that don't have AIA built in?

[timsnyder]

When I try to download the source tarball from dakota.sandia.gov (e.g. https://dakota.sandia.gov/sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz ), I'm able to download fine if I use a full-fledged browser like Chrome or Firefox to do the download. However, when using curl or python, I encounter SSL validation errors. I'm using the root certs from 2022.09 version of certifi.

This is problematic when trying to build dakota in an automated workflow as in https://github.com/conda-forge/dakota-feedstock. Looking for other package recipes in repology, I don't see other packagers struggling with this but there aren't that many who package dakota and are up to date with at least 6.16.

How to reproduce SSL error

-bash-4.2$ which python
~/.conda/envs/aia/bin/python
-bash-4.2$ conda list -n aia
# packages in environment at /home/centos/.conda/envs/aia:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aia                       0.2.0                    pypi_0    pypi
brotlipy                  0.7.0           py311hd4cff14_1005    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
ca-certificates           2022.9.24            ha878542_0    conda-forge
certifi                   2022.9.24          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py311h409f033_2    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
cryptography              38.0.3          py311h42a1071_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
ld_impl_linux-64          2.39                 hc81fddc_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libsqlite                 3.40.0               h753d276_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
openssl                   3.0.7                h166bdaf_0    conda-forge
pip                       22.3.1             pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 22.1.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.11.0          ha86cf86_0_cpython    conda-forge
python_abi                3.11                    2_cp311    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
requests                  2.28.1             pyhd8ed1ab_1    conda-forge
setuptools                65.5.1             pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
tzdata                    2022f                h191b570_0    conda-forge
urllib3                   1.26.11            pyhd8ed1ab_0    conda-forge
wheel                     0.38.4             pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
-bash-4.2$ cat get_wo_aia.py 
#!/usr/bin/env python

import requests

url = "https://dakota.sandia.gov/sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz"
outfile = "./dakota-6.16.0-public-src-cli.tar.gz"

resp = requests.get(url, stream=True)
with open(outfile, 'wb') as fd:
    for chunk in resp.iter_content(chunk_size=128):
        fd.write(chunk)
print(f"wrote {outfile}")
-bash-4.2$ ./get_wo_aia.py 
Traceback (most recent call last):
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
                ^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/ssl.py", line 1075, in _create
    self.do_handshake()
  File "/home/centos/.conda/envs/aia/lib/python3.11/ssl.py", line 1346, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dakota.sandia.gov', port=443): Max retries exceeded with url: /sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/centos/dakota-feedstock/recipe/./get_wo_aia.py", line 8, in <module>
    resp = requests.get(url, stream=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/centos/.conda/envs/aia/lib/python3.11/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='dakota.sandia.gov', port=443): Max retries exceeded with url: /sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)')))

If I use the pypi aia library to fetch intermediate certificates using Authority Information Access (AIA) extension, then downloading in Python works.

Using aia library to build full cert chain

-bash-4.2$ diff -U3 get_wo_aia.py get_w_aia.py 
--- get_wo_aia.py	2022-11-19 20:33:07.801375404 +0000
+++ get_w_aia.py	2022-11-19 20:40:12.098807804 +0000
@@ -1,12 +1,20 @@
 #!/usr/bin/env python
 
+from tempfile import NamedTemporaryFile
+from aia import AIASession
 import requests
 
+aia_session = AIASession()
 url = "https://dakota.sandia.gov/sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz"
 outfile = "./dakota-6.16.0-public-src-cli.tar.gz"
 
-resp = requests.get(url, stream=True)
-with open(outfile, 'wb') as fd:
-    for chunk in resp.iter_content(chunk_size=128):
-        fd.write(chunk)
-print(f"wrote {outfile}")
+cadata = aia_session.cadata_from_url(url)  # Validated PEM certificate chain
+#print(f"cadata:\n{cadata}\n\n")
+with NamedTemporaryFile("w") as pem_file:
+    pem_file.write(cadata)
+    pem_file.flush()
+    resp = requests.get(url, verify=pem_file.name, stream=True)
+    with open(outfile, 'wb') as fd:
+        for chunk in resp.iter_content(chunk_size=128):
+            fd.write(chunk)
+    print(f"wrote {outfile}")
-bash-4.2$ cat get_w_aia.py 
#!/usr/bin/env python

from tempfile import NamedTemporaryFile
from aia import AIASession
import requests

aia_session = AIASession()
url = "https://dakota.sandia.gov/sites/default/files/distributions/public/dakota-6.16.0-public-src-cli.tar.gz"
outfile = "./dakota-6.16.0-public-src-cli.tar.gz"

cadata = aia_session.cadata_from_url(url)  # Validated PEM certificate chain
#print(f"cadata:\n{cadata}\n\n")
with NamedTemporaryFile("w") as pem_file:
    pem_file.write(cadata)
    pem_file.flush()
    resp = requests.get(url, verify=pem_file.name, stream=True)
    with open(outfile, 'wb') as fd:
        for chunk in resp.iter_content(chunk_size=128):
            fd.write(chunk)
    print(f"wrote {outfile}")
-bash-4.2$ ./get_w_aia.py 
wrote ./dakota-6.16.0-public-src-cli.tar.gz
-bash-4.2$ sha256sum dakota-6.16.0-public-src-cli.tar.gz 
49684ade2a937465d85b0fc69c96408be38bc1603ed2e7e8156d93eee3567d2f  dakota-6.16.0-public-src-cli.tar.gz

And the checksum matches what I downloaded interactively with a browser.

https://unix.stackexchange.com/questions/559526/downloading-from-a-public-server-ssl-certificate-problem-unable-to-get-local-i is a decent description of AIA fetching.

I can script around this problem for the conda-forge package recipe but if the configuration of dakota.sandia.gov could be updated to provide the full certificate chain I'd really appreciate it. Thanks.

[briadam]

Hi Tim -- Our sys admin is looking into this. -- Brian

[briadam]

@timsnyder the server hosting dakota.sandia.gov should now be configured with the full certificate chain. We've done some verification using wget and curl from outside sandia.gov, but let us know what your workflow now finds.

[timsnyder]

Thanks @briadam. Seems to be working now and I've been able to revert my workaround.