HTTP operator with custom certificate in python bindings

[bennyz]

I am trying to use the http operator in the python bindings to access a url with a custom certificate, but I get:

error sending request for url (https://internal/index.html): client error (Connect): invalid peer certificate: UnknownIssuer

with:

python3 -c "from opendal import Operator; print(Operator('http', endpoint='https://internal').read('index.html').decode().splitlines()[0])"

I have added the certificate to the system, and it works with curl, but I can't get it to work. It seems to be related to the features enabled by reqwest, but I guess I can't control from the python library? Is there any way around this?

[dosubot[bot]]

Hi @bennyz! I'm Dosu and I’m helping the opendal team.

This happens because OpenDAL's Python bindings use Rust's reqwest library for HTTP, which relies on its own certificate store and doesn't always use the system or curl trust store. Right now, the Python bindings don't expose any way to configure custom CA bundles or disable SSL verification for the HTTP operator—these options are only available in the underlying Rust API, not in Python.

Workarounds include using a reverse proxy that handles TLS for you, or (if you're comfortable with Rust) modifying the Rust layer to disable verification for development purposes. If this feature is critical, you may want to track or request support for custom certificate configuration in the Python bindings, as it's a known limitation.

Reference: relevant maintainer discussion

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[bennyz]

further investigation shows the issue seems to be specific to operator#stat