docs(arrow-layers): Clean up arrow-layers docs and file structure (#69)

Author: ibgreen

docs/README.md

@@ -1,30 +1,42 @@
 # Introduction
 
-This repository contains a collection of community supported modules for [deck.gl](https://deck.gl), that are intended to complement the various modules already provided by the core deck.gl framework. While any module that is properly scoped and of sufficient value to the community could be a candidate for this repository, common modules type are:
+This repository contains a collection of community supported modules for [deck.gl](https://deck.gl).
+It was initially created to provide a home for a number of excellent deck.gl add-on modules that had fallen into disuse.
+
+## Scope
+
+This repository is intended to host modules that complement the various modules already provided by the core deck.gl framework. 
+While any module that is properly scoped and of sufficient value to the community could be a candidate for this repository, 
+common modules type are:
 
 - additional **layer packs** (beyond the various layer packs available in deck.gl)
 - additional **base map** integrations (beyond the integrations supported by deck.gl)
-- additional **react** bindings (beyond the `@deck.gl/react` module).
+- additional **React bindings** (beyond the `@deck.gl/react` module).
 
-## Support
+## Contributing
 
-Community modules are not officially supported by the core deck.gl maintainers, but have at least some intermittent, part-time support from one or more community members.
+For extensions to existing modules, it is generally recommended to start a discussion before you open a PR.
 
-Note that the continued inclusion of each module into this repository depends to a large extent on whether there is sufficient community support for the module. Modules can be removed from this repository if the core deck.gl team feels that the community is no longer able to provide sufficient support.
+If you have a new module that you think could fit into this repository, please start by opening a GitHub issue to start a discussion, or reach out in the OpenJS slack.
+Note that for a new module you will also be asked to asses what level of maintenance you will be able to provide over the longer term.
 
-If a module was to be removed, applications can of course copy the module's source code, but will need to maintain the code on their own.
+## Governance
 
-## Contributing
+Final decision ultimately rest with the OpenJS Open Visualization TSC (Technical Steering Committee), but decisions are often made in the open bi-weekly meetings.
 
-If you have a module that you think could fit into this repository, please start by opening a GitHub issue to start a discussion, or reach out in the OpenJS slack.
+## Support
 
-## Upstreaming
+Community modules are not officially supported by the core deck.gl maintainers,
+but are expected to have at least intermittent, part-time support from one or more community members.
 
-On rare occasions, a new component or module may be "upstreamed" into the core deck.gl repository. 
+Overall goals for this repo is
+- All modules should support deck.gl v9 on WebGL2.
+- Modules will be expected to gradually start supporting deck.gl v9 on WebGPU 
+- Support for deck.gl v8 is a non-goal, though one or two modules may have older versions that still work.
 
-There is a high bar when adding new code to the main deck.gl repository. The deck.gl-community repository is sometimes used to prepare (incubate) new software components so that they are ready to be added to deck.gl. 
+## Insufficient Support
 
-Therefore when proposing the addition of a new component (such as a new deck.gl layer),
-to the core deck.gl maintainers, it is helpful to be able to prepare the component in a monorepo environment that is similar to the deck.gl repo, complete with tests, documentation and examples. This can avoid a length and frustrating review process in the deck.gl repo.
+Note that the continued inclusion of each module into this repository depends to a large extent on whether there is sufficient community support for the module. 
+Modules can be removed from this repository if the core deck.gl team feels that the community is no longer able to provide sufficient support.
 
-To set expectations, most components in this repository will never be added to the main deck.gl repository.
+If a module was to be removed, applications can of course copy the module's source code, but will need to maintain the code on their own.

docs/modules/arrow-layers/README.md

@@ -0,0 +1,31 @@
+# Overview
+
+:::danger
+The arrow layers module is still in the process of being ported into deck.gl-community from [geoarrow/deck.gl-layers](https://github.com/geoarrow/deck.gl-layers).
+:::
+
+This module provides deck.gl layers that accept Apache Arrow and [GeoArrow](https://geoarrow.org) tables. 
+These layers take advantage of the deck.gl [low-level binary interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly) to provide binary arrow data from Apache Arrow tables directly to the GPU.
+
+
+![arrow-layers example](./images/hero.jpg)
+<p style={{textAlign:"center"}}>3.2 million points rendered with a <code>GeoArrowScatterplotLayer</code>.</p>
+
+## Features
+
+- **Fast**: copies binary buffers directly from an [Arrow JS](https://www.npmjs.com/package/apache-arrow) [`Table`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html) object to the GPU using [deck.gl's binary data interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly).
+- **Memory-efficient**: no intermediate data representation and no garbage-collector overhead.
+- **Full layer customization**: Use the same layer properties as in the upstream deck.gl layer documentation. Any _accessor_ (layer property prefixed with `get*`) can be passed an Arrow [`Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).
+- **Input validation**. Validation can be turned off via the `_validate` property on most layer types.
+- **Multi-threaded polygon triangulation**. When rendering polygon layers, a process called [polygon triangulation](https://en.wikipedia.org/wiki/Polygon_triangulation) must happen on the CPU before data can be copied to the GPU. Ordinarily, this can block the main thread for several seconds, but the `GeoArrowSolidPolygonLayer` will perform this process off the main thread, on up to 8 web workers.
+- **Progressive rendering support**. For streaming-capable data formats like Arrow IPC and Parquet, you can render a GeoArrow layer per chunk as the data loads.
+
+## Examples
+
+Standalone examples exist in the [`examples/`](examples/) directory. Create an issue if you have trouble running them.
+
+## History
+
+This module is based on the [geoarrow/deck.gl-layers](https://github.com/geoarrow/deck.gl-layers) module, originally developed by Kyle Barron as part of his independent multi-prong effort to implement fully optimized GeoArrow and GeoParquet based browser visualizations. 
+
+It was agreed that the deck.gl-community repository is a good long-term home for this particular module. Of note is that there is also interest to bring Apache Arrow support into the deck.gl core repository in a future release and therefore moving development of these layers closer to deck.gl will reduce the risk of duplicated work.

docs/modules/arrow-layers/api-reference/layers.md

@@ -0,0 +1,47 @@
+# Layers
+
+The following layers are provided, offer Apache Arrow bindings to the corresponding official deck.gl layers
+
+## GeoArrowArcLayer
+
+TBA
+
+## GeoArrowColumnLayer
+
+TBA
+
+## GeoArrowH3HexagonLayer
+
+TBA
+
+## GeoArrowHeatmapLayer
+
+TBA
+
+## GeoArrowPathLayer
+
+TBA
+
+## GeoArrowPointCloudLayer
+
+TBA
+
+## GeoArrowPolygonLayer
+
+TBA
+
+## GeoArrowScatterplotLayer
+
+TBA
+
+## GeoArrowSolidPolygonLayer
+
+TBA
+
+## GeoArrowTextLayer
+
+TBA
+
+## GeoArrowTripsLayer
+
+TBA

docs/modules/README.md renamed to docs/modules/arrow-layers/developer-guide/get-started.md

@@ -1,27 +1,4 @@
-# @deck.gl-community/arrow-layers
-
-The easiest, most efficient way to render large geospatial datasets in [deck.gl](https://deck.gl), via [GeoArrow](https://geoarrow.org).
-
-This is just a _glue library_ to deck.gl. It generates the same layer objects as upstream deck.gl does, but uses a [low-level binary interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly) for best performance. Using the binary interface directly is really easy to mess up. Instead, the layer classes exposed by `@deck.gl-community/arrow-layers` focus on making the process easy to use and validating user input, and under the hood pass buffers to deck.gl's binary interface.
-
-![](assets/hero.jpg)
-
-<p style="text-align:center">3.2 million points rendered with a <code>GeoArrowScatterplotLayer</code>.</p>
-
-## Features
-
-- **Fast**: copies binary buffers directly from an [Arrow JS](https://www.npmjs.com/package/apache-arrow) [`Table`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Table.html) object to the GPU using [deck.gl's binary data interface](https://deck.gl/docs/developer-guide/performance#supply-attributes-directly).
-- **Memory-efficient**: no intermediate data representation and no garbage-collector overhead.
-- **Full layer customization**: Use the same layer properties as in the upstream deck.gl layer documentation. Any _accessor_ (layer property prefixed with `get*`) can be passed an Arrow [`Vector`](https://arrow.apache.org/docs/js/classes/Arrow_dom.Vector.html).
-- **Input validation**. Validation can be turned off via the `_validate` property on most layer types.
-- **Multi-threaded polygon triangulation**. When rendering polygon layers, a process called [polygon triangulation](https://en.wikipedia.org/wiki/Polygon_triangulation) must happen on the CPU before data can be copied to the GPU. Ordinarily, this can block the main thread for several seconds, but the `GeoArrowSolidPolygonLayer` will perform this process off the main thread, on up to 8 web workers.
-- **Progressive rendering support**. For streaming-capable data formats like Arrow IPC and Parquet, you can render a GeoArrow layer per chunk as the data loads.
-
-## Examples
-
-Standalone examples exist in the [`examples/`](examples/) directory. Create an issue if you have trouble running them.
-
-More hosted examples on Observable are planned.
+# Get Started
 
 ## Providing accessors
 
@@ -66,7 +43,7 @@ const deckLayer = new GeoArrowPathLayer({
 }),
 ```
 
-The full example is in [`examples/multilinestring/app.tsx`](examples/multilinestring/app.tsx).
+The full example is in `examples/multilinestring/app.tsx`.
 
 You can also use assign to the `target` prop to reduce garbage collector overhead, as described in the [deck.gl performance guide](https://deck.gl/docs/developer-guide/performance#supply-binary-blobs-to-the-data-prop).
 

docs/modules/arrow-layers/sidebar.json

@@ -0,0 +1,9 @@
+{
+  "type": "category",
+  "label": "@deck.gl-community/arrow-layers",
+  "items": [
+    "modules/arrow-layers/README",
+    "modules/arrow-layers/developer-guide/get-started",
+    "modules/arrow-layers/api-reference/layers"
+  ]
+}

docs/modules/editable-layers/README.md

@@ -18,7 +18,7 @@ Provides editable and interactive map overlay layers, built using the power of [
 
 ## History
 
-A fork of @nebula.gl. nebula.gl is an important part of the deck.gl ecosystem but the repository has lacked maintainers for several years and the repository no longer accepts external contributions.
+A fork of @nebula.gl. [nebula.gl](https://nabula.gl) is an important part of the deck.gl ecosystem but the repository has lacked maintainers for several years and the repository no longer accepts external contributions.
 
 ## What's New
 
@@ -38,13 +38,6 @@ This page contains highlights of each `editable-layers` release.
 | `@nebula.gl/editor`                       | React wrappers      | => `@deck.gl-community/react-editable-layers` |
 | `react-map-gl-draw`                       | Non-deck-wrapper    | => NOT FORKED                                 |
 
-
-### editable-layers v0.0.1
-
-Release date: TBD
-
-- new `DrawRectangleFromCenterMode`. User can draw a new rectangular `Polygon` feature by clicking the center, then along a corner of the rectangle.
-- `screenSpace` option can be provided in the `modeConfig` of Translate mode so the features will be translated without distortion in screen space.
-- `lockRectangles` option can be provided in the `modeConfig` object for ModifyMode, so the features with `properties.shape === 'Rectangle'` will preserve rectangular shape.
-- `pickingLineWidthExtraPixels` property to specify additional line width in pixels for picking. Can be useful when `EditableGeojsonLayer` is over a deck.gl layer and precise picking is problematic, and when usage of `pickingDepth` introduces performance issues.
-
+Notes:
+- `react-map-gl-draw`- A notable omission is that `react-map-gl-draw` is not included in this fork. This decision was made to simplify the nebula.gl code base with the hope of making it easier for non-dedicated maintainers to keep the deck.gl layers version of nebula.gl alive. Given that the new version is no longer broken into deck.gl independent modules, it may not be easy to add.
+The main reason for updating react-map-gl-draw is likely to make it work with the latest React versions. If it helps, `react-map-gl-draw` is a small module and you can probably copy the source code into your app and bump the react dependency.

docs/modules/editable-layers/developer-guide/old.md

@@ -5,17 +5,19 @@
 
 `graph-layers` is a deck.gl layer pack for GPU-powered visualization of large graphs. 
 
-With graph-layers, developers can build various type of graph/network applications with minimum effort. The composable API enables highly customizable graph visualization by leveraging or even extending the provided graph *styles* and *layout algorithms*.
+With graph-layers, developers can build various type of graph/network applications with minimum effort. 
+The composable API enables highly customizable graph visualization by leveraging or even extending the provided graph *styles* and *layout algorithms*.
 
 ## History
 
-`graph-layers` started out as a friendly fork of Uber's [graph.gl](https://graph.gl/gatsby/) framework. 
+`graph-layers` started out as a friendly fork of Uber's archived [graph.gl](https://graph.gl/gatsby/) framework, 
+bringing the graph.gl code up-to-date with the latest deck.gl versions. 
 
 ## What's New
 
-### `@deck.gl-community/graph-layers` v9.0.0
+### `@deck.gl-community/graph-layers` v9.0.0 (In development)
 
-Release date: April 2024
+Target Release date: April 2024
 
 The graph-layers module has been repackaged as a deck.gl "layer pack" with additional features, and has been modernized in terms of: 
 - upgraded to work with deck.gl v9 
@@ -25,6 +27,6 @@ The graph-layers module has been repackaged as a deck.gl "layer pack" with addit
 
 ### `deck-graph-layers` v0.0.1
 
-Release date: Q4 2023
+Release date: April 14, 2023
 
 An initial fork of Uber's [graph.gl](https://github.com/uber/graph.gl) repository. At the time of the fork, the repository has lacked maintainers for several years, the published versions were not compatible with recent deck.gl versions, and the repository no longer accepted external contributions.

docs/modules/graph-layers/README.md

@@ -1,4 +1,8 @@
 # Overview
 
+:::caution
+There is an ambition to phase out React specific code for deck.gl-community modules. This model is expected to be deprecated in the near future.
+:::
+
 react-graph-layers provides React components for visualizing large graphs with several utility functions. It can build a highly customizable graph visualization through its composable API. The rendering is powered by deck.gl which is a WebGL based visualization framework.  With react-graph-layers, users are enabled to build various type of graph/network applications with minimum efforts while having the capability to extend the existing styles and layouts.
 

docs/modules/react-graph-layers/README.md

@@ -1,39 +1,18 @@
 # What's New
 
-The following releases are deck.gl v9 compatible
+The detailed release notes of each module can be found in the module-specific docs section.
 
-### `@deck.gl-community/graph-layers` v9.0.0
+High-level updates are 
 
-Target Release date: April 2024
 
-The deck-graph-layers module is being repackaged as a deck.gl "layer pack" and modernized
+April 15, 2024: [**`@deck.gl-community/editable-layers`**](/docs/modules/editable-layers)) v9 - This new layer pack is a fork of Uber's no longer maintained [nebula.gl](https://nebula.gl) framework. nebula.gl has been an important part of the deck.gl ecosystem but the repository has lacked maintainers for several years and the repository no longer accepts external contributions.
 
 
-### `@deck.gl-community/editable-layers` v9.0.0
+Feb 29, 2024: [**`@deck.gl-community/layers`**](/docs/modules/layers) v9 - deck,gl community-layers now support deck.gl v9.
 
-Release date: April 15, 2024
 
-This new module is a fork of Uber's no longer maintained @nebula.gl framework.is an important part of the deck.gl ecosystem but the repository has lacked maintainers for several years and the repository no longer accepts external contributions.
+December 22, 2023: [**`@deck.gl-community/layers`**](/docs/modules/layers) v0 - A new module intended to containing a collection of useful community layers. Initial layers are `TileSourceLayer`, `DataDrivenTile3DLayer`.
 
 
-### `@deck.gl-community/layers` v9.0.0
+April 14, 2023: [**`@deck-graph-layers`**](/docs/modules/graph-layers) - A new layer pack for rendering graphs (nodes and edges). Forked from Uber's archived [graph.gl](https://graph.gl) repo.
 
-- Target Release date: March 26, 2024
-- New version of the `@deck.gl-community/layers` module for deck.gl v9.
-
----
-
-Releases below are deck.gl v8 compatible
-
-> Support for the v8 ecosystem is not a goal, anything below is available strictly on an as-is basis.
-
-### `@deck.gl-community/layers` v0.0.0
-
-Release date: 2023
-
-- `TileSourceLayer`
-- `DataDrivenTile3DLayer`
-
-### `@deck-graph-layers` v0.0.1 (NEW MODULE)
-
-A new layer pack built from a fork of Uber's graph.gl. For more information see [`graph-layers`](/docs/modules/graph-layers).