How to load local shapefile into a JupyterGIS project?

[matthewfeickert]

Note

This is almost certainly an out of scope request, as this is not something that is JupyterGIS specific, but a question about interacting with open data through JupyterGIS. So feel free to close as out of scope without any explanation.

It would be nice to add an example to the docs that shows how to explore open data sets, which might exist as shapefiles that aren't already in the QGIS format (this is the probably out of scope part).

Specific example that I have

The National Ecological Observatory Network (NEON) publishes all the ecological data it produces, including spatial data as shapefiles. As an example to learn how to play around with open data and JupyterGIS I first wanted to open the 2024 scientific domain shapefile

First I created an environment with pixi

pixi init
pixi add jupytergis qgis curl
pixi shell

resulting manifest:

[project]
authors = ["Matthew Feickert <matthew.feickert@cern.ch>"]
channels = ["conda-forge"]
description = "Add a short description here"
name = "neon_gis_data"
platforms = ["linux-64"]
version = "0.1.0"

[tasks.download]
description = "Download NEON domains from https://www.neonscience.org/data-samples/data/spatial-data-maps"
# c.f. https://www.neonscience.org/data-samples/data/spatial-data-maps
cmd = """
mkdir -p data && \
cd data && \
curl -sLO https://www.neonscience.org/sites/default/files/NEONDomains_2024.zip && \
curl -sLO https://github.com/geojupyter/jupytergis/raw/refs/heads/main/examples/buildings.qgz
"""

[dependencies]
jupytergis = ">=0.2.0,<0.3"
qgis = ">=3.40.2,<4"
curl = ">=8.11.1,<9"

or with conda that would be

conda create --name example --yes jupytergis qgis curl
conda activate example

so now with either tool we're in an activated environment.

I then downloaded the zipped shapefile

mkdir -p data && cd data
curl -sLO https://www.neonscience.org/sites/default/files/NEONDomains_2024.zip

and for comparison we'll also grab the very nicely provided example buildings QGIS file from jupytergis

curl -sLO https://github.com/geojupyter/jupytergis/raw/refs/heads/main/examples/buildings.qgz

However, the NEON shapefile isn't already a QGIS format and so it needs to be converted. I don't know the qgis Python API and as searching and poking at LLM attempts at it did fairly poorly I decided to give up and use the QGIS GUI and to convert the zipped shapefile to a vector layer and then save the project.

Commands/actions to do so:

1. Launch gui from command line

qgis

2. Open zipped shapefile as vector layer:
   • "Layer" menu -> "Add Layer" -> "Add Vector Layer"
   • Source Type: File
   • Encoding: Automatic
   • Source: NEONDomains_2024.zip
   • Click "Add" button

3. Save project state as a QGIS project (.qgz) file:

   • "Project" menu -> "Save As"
   • Ensure under "data" dir
   • File name: neon_domains_2024
   • • Click "Save" button

4. Exit QGIS gui.

If I then launch Jupyter Lab (after going one directory up)

cd ..
jupyter lab

and create a new notebook named example.ipynb

# Cell 1
from jupytergis_lab import GISDocument
from pathlib import Path

# Cell 2
example_path = Path.cwd() / "data" / "neon_domains_2024.qgz"
doc = GISDocument(str(example_path))

doc

the notebook executes without error but it just shows a blank canvas with an interactive distance scale in the bottom left

If I change to using the jupytergis example buildings file

example_path = Path.cwd() / "data" / "buildings.qgz"

then things work as expected, both in the notebook and using the JupyterLab Extension. That's great of course, as jupytergis is doing an amazing thing, but I'd like to explore more data.

Motivation for showing open data examples

My assumption is that I have messed up the conversion from the shapefile to QGIS. This is a "me problem" and outside the jupytergis project.

However, I think it would be generally useful to show interactions with open data examples (that of course have some long term support so that the examples don't break all the time) as my uninformed assumption is that most open data shapefiles are not (yet) in QGIS format. So providing at least one example, or linking to external documentation, on how to interact with non-QGIS files could be helpful.

[mfisher87]

Currently, we don't support opening a local shapefile as a layer, but we will! GeoJSON is supported at the moment, though, so I'd recommend converting your shapefile to geojson and then opening it:

ogr2ogr -f GeoJSON output.json input.shp

(Yes, the positional args are out of order, I don't know why 🤕

I think you can pass in the zip file directly instead of uncompressing it first, give it a shot :)

You can also do this entirely in QGIS.

GeoJSON is the only local vector file format that JupyterGIS currently supports.

The reason the method you attempted wasn't working is because .qgz or .qgs files are QGIS project files, not data files, equivalent to JupyterGIS' .jgis files. QGIS project files are structured XML, and JGIS project files are structured JSON. Neither actually have data baked in to them. Instead they have information like layer structure and symbology information, and they link to the data that the project uses, either as URLs or local file paths. So, when you loaded the shapefile into QGIS and saved the .qgz, it only contained a reference to the shapefile.

Finally, I really like this website ;) http://switchfromshapefile.org/

QuantStack folks, did I miss anything important?

[mfisher87]

Interesting! We'd be very appreciative if you could open an issue! Thanks so much for offering and for providing such thorough detail ❤️

[matthewfeickert]

Will do. By the way, @mfisher87, I assume that you were testing with a local install and serve of JupyterLite? On the current deployment of https://geojupyter.github.io/jupytergis/lite/lab/index.html piplite and micropip aren't available so one can't install pure-Python wheels from PyPI with the

import piplite

await piplite.install(["target-package-name", "additional-install-package-name"])
import target_pacakge_name

pattern.

[matthewfeickert]

Issue opened in #318.

[mfisher87]

I assume that you were testing with a local install and serve of JupyterLite? On the current deployment of geojupyter.github.io/jupytergis/lite/lab/index.html piplite and micropip aren't available so one can't install pure-Python wheels from PyPI

I was using our ReadTheDocs deployment of JupyterGIS 0.2.0 in JupyterLite if I remember correctly. I'm really new to the WASM world 🫣 so I must be missing something important about your question -- why do you ask?

[matthewfeickert]

(Sorry, slow reply. 😬)

I'm really new to the WASM world 🫣 so I must be missing something important about your question -- why do you ask?

Totally fine. :) I know only how to use things, not develop for it or build WASM wheels. My reason for asking is just curiosiity and basically Issue #342.