Ambiguity in installing duckdb for pyodide/pyscript

[ttomasz]

When I include DuckDB as dependency in PyScript it installs v1.1.2 via micropip from URL:
https://cdn.jsdelivr.net/pyodide/v0.27.2/full/duckdb-1.1.2-cp312-cp312-pyodide_2024_0_wasm32.whl

Then I found this repo. It would be nice if you could coordinate with Pyodide folks to have one distribution of DuckDB instead of multiple.

Or at least it would be nice if docs clarified the situation and explained how and why one might want to install each version when using Pyodide.

[agriyakhetarpal]

I've tried to clarify some things in #6 (comment). I'd like to note that we haven't supported past Pyodide versions at all; maintenance is always tough :) At least from the Pyodide side, we support only the latest Pyodide version for bugs and feature requests. Still, it is probably okay and actually rather nice from downstream packagers like the DuckDB team to support multiple Pyodide versions.

We try to document ABI breaks and the underlying Python version in the CHANGELOG as much as we can. There is one in the upcoming Pyodide version 0.28 later this year – it will build against Emscripten 4.0.1 and bring some changes on the Rust side, as WASM exception handling was introduced (pyodide/pyodide-build#81), which required a custom Rust toolchain. Downstream libraries must switch from -fexceptions to -fwasm-exceptions.

At the time, a small note in the README here which relays over this information from Pyodide for ease of access could be a good start.

[ttomasz]

@agriyakhetarpal maybe there is a little confusion here but I'm not talking about different pyodide versions. There are two separate distributions of duckdb package:

• this repo: https://duckdb.github.io/duckdb-pyodide/wheels/duckdb-1.2.0-cp312-cp312-pyodide_2024_0_wasm32.whl
• i guess build from pyodide team: https://cdn.jsdelivr.net/pyodide/v0.27.2/full/duckdb-1.1.2-cp312-cp312-pyodide_2024_0_wasm32.whl

this is confusing as a user.

[agriyakhetarpal]

Oh, that makes much more sense to me now. I'm sorry, @ttomasz! The DuckDB team has added new wheels here from time to time, but they are indeed not updated in Pyodide's in-tree recipe as often. For the record, it looks like https://duckdb.github.io has CORS headers set. You may use

import micropip; await micropip.install("https://duckdb.github.io/duckdb-pyodide/wheels/duckdb-1.2.0-cp312-cp312-pyodide_2024_0_wasm32.whl")

to install a particular DuckDB wheel, whereas await micropip.install("duckdb") will likely get you a slightly outdated version from the Pyodide CDN (until we complete our work on unvendoring recipes from the Pyodide distribution). I hope this helps!