Questions RE: PyDeck GeoJson Layer Data Format (URL, JSON, Dataframe)

[KeeonTabrizi]

TL;DR Questions

• What is the structure of a dataframe that can be passed to a GeoJsonLayer? Assuming a record per feature and each feature could have properties?
• Is it expected that a external URL of geojson would perform better in the browser than loading a local json file (see example below).

---

Observations

I have been looking at documentation / code from PyDeck 0.8.3b, specifically the examples using the GeoJsonLayer

• https://deckgl.readthedocs.io/en/latest/gallery/geojson_layer.html
• https://github.com/visgl/deck.gl/blob/master/bindings/pydeck/examples/geojson_layer.py
• https://deck.gl/docs/api-reference/layers/geojson-layer

Additionally, based on my read of some of the underlying code of the python API bindings to deck.gl it states

data : str or list of dict of {str: Any} or pandas.DataFrame, default None
Either a URL of data to load in or an array of data

• deck.gl/bindings/pydeck/pydeck/bindings/layer.py

  Line 34 in 1a42362

  data : str or list of dict of {str: Any} or pandas.DataFrame, default None

• deck.gl/bindings/pydeck/pydeck/data_utils/type_checking.py

  Line 16 in 1a42362

  def records_from_geo_interface(data):

The examples that point to a URL of the (geo)json work fine. In fact even in my tests of pointing them to much larger files they also render peformantly (as in the map renders and can pan smoothly). What is a bit strange to me is how the performance drops a lot when taking the same file and loading it as json.

Example:

import pydeck as pdk
import json
     with open('/path/to/example.json') as f:
                    data = json.load(f) # note that the 'features' key can also be selected and it works
pdk.Layer("GeoJsonLayer", data, ...)
pdf.Deck(...)

Both the example of using the DATA_URL and this method to load a local file work. The former being more performant in my testing. For kicks I tried turning the local file into a URI (i.e. file:///path/to/fexample.json) and that would not render.

Loading the (geo)json also will work if you filter the data json object to data['features'].

One thing I don't currently understand is how to pass in a dataframe to the geojson layer. In the GeoJson you have an array of features which will have at minimum a geometry key as well as an optional properties key.

[kylebarron]

For kicks I tried turning the local file into a URI (i.e. file:///path/to/fexample.json) and that would not render

Chrome doesn't allow sites to perform requests to file://

If you want to load local data, you need to host the local data on a server on localhost, e.g. with python -m http.server

[KeeonTabrizi]

For kicks I tried turning the local file into a URI (i.e. file:///path/to/fexample.json) and that would not render

Chrome doesn't allow sites to perform requests to file://

If you want to load local data, you need to host the local data on a server on localhost, e.g. with python -m http.server

Makes sense, though I found it strange instead of an external URL reference to a file I couldn't point to a path to a local file, instead I loaded the file (example json loading I showed), and saw notable performance degradation as I mentioned.

In any case, I've been still trying to figure out the dataframe question too :) !

[kylebarron]

• Is it expected that a external URL of geojson would perform better in the browser than loading a local json file

Yeah, could be, because in the first case the data never goes through Python, but rather Python sends JS just a short string. Whereas in the second case all of the data goes through Python, and Python needs to send the entire JSON-encoded data from the Python side to the JS side

[kylebarron]

• What is the structure of a dataframe that can be passed to a GeoJsonLayer?

I assume you can pass in a geopandas GeoDataFrame but I haven't looked lately

[KeeonTabrizi]

Okay! I figured out the dataframe part.

It's straightforward - Basically you just need a column geometry which will hold the geometry dict of the underlying feature element from a geojson object.

Every other column in the dataframe is treated also loaded akin to the properties in geojson.

The trick was making sure the dataframe geometry column was loaded as dict.

import pandas as pd
import ast

# on load
df = pd.read_csv('path/to/file.csv', converters = {'geometry': ast.literal_eval})
# or convert column of already load dataframe
try:
    df_existing['geometry'] =  df_existing['geometry'].apply(ast.literal_eval) # is there a better way?
except ValueError:  # maybe already converted object to dict previously
    pass

Under the hood pydeck takes this dataframe and calls df.to_dict(orient='records') - this structure of data can also be passed.

Generally, I'm still find that just pointing to a URL (when it's geojson) seems to out perform the other methods.

[kylebarron]

{'geometry': ast.literal_eval})

Yikes! It's a lot easier to do

df['geometry'] = df['geometry'].apply(json.loads)

This is also a lot easier to do in geopandas https://geopandas.org/en/stable/

[KeeonTabrizi]

Yeah makes sense - I think I had some rogue single versus double quotes where that more archaic method :) was working.