Skip to content

Commit a2d670a

Browse files
authored
Aggregate API docs for flat modules (#648)
* Define `__all__` for `movement.roi` * Manually generate autosummary pages for public api modules * Temporarily enable artifactci for preview * Fix module names in `exclude_modules` * Use jinja2 templating * Refactor `make_api_index.py` * Update module descriptions * Rename `make_api` script * Update CONTRIBUTING.md and make_api.py descriptions * Fix reference to `compute_velocity()` in `movement_dataset.md` * Add BaseRegionOfInterest to `roi` module exports * Return sorted functions and classes in get_members() * Update doc references in `arc-collaboration.md` * Update references to `BaseRegionOfInterest` in docstrings * Revert "Temporarily enable artifactci for preview" This reverts commit b2b49c7.
1 parent f6fd691 commit a2d670a

File tree

16 files changed

+162
-72
lines changed

16 files changed

+162
-72
lines changed

CONTRIBUTING.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -248,13 +248,18 @@ If you are adding references to an external URL (e.g. `https://github.com/neuroi
248248
If it is not yet defined and you have multiple external URLs pointing to the same base URL, you will need to [add the URL scheme](myst-parser:syntax/cross-referencing.html#customising-external-url-resolution) to `myst_url_schemes` in `docs/source/conf.py`.
249249

250250
### Updating the API reference
251-
The [API reference](target-api) is auto-generated by the `docs/make_api_index.py` script, and the [sphinx-autodoc](sphinx-doc:extensions/autodoc.html) and [sphinx-autosummary](sphinx-doc:extensions/autosummary.html) extensions.
252-
The script generates the `docs/source/api_index.rst` file containing the list of modules to be included in the [API reference](target-api).
253-
The plugins then generate the API reference pages for each module listed in `api_index.rst`, based on the docstrings in the source code.
251+
The [API reference](target-api) is auto-generated by the `docs/make_api.py` script, and the [sphinx-autodoc](sphinx-doc:extensions/autodoc.html) and [sphinx-autosummary](sphinx-doc:extensions/autosummary.html) extensions.
252+
The script inspects the source tree and generates the `docs/source/api_index.rst` file, which lists the modules to be included in the [API reference](target-api), skipping those listed in `EXCLUDE_MODULES`.
253+
254+
For each _package module_ listed in `PACKAGE_MODULES`—a module that re-exports selected classes and functions from its submodules via `__init__.py` (e.g. {mod}`movement.kinematics`)—the script also generates a `.rst` file in `docs/source/api/` with autosummary entries for the top-level objects exposed by the module.
255+
256+
The Sphinx extensions then generate the API reference pages for each module listed in `api_index.rst`, based on their docstrings.
254257
So make sure that all your public functions/classes/methods have valid docstrings following the [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html) style.
255258
Our `pre-commit` hooks include some checks (`ruff` rules) that ensure the docstrings are formatted consistently.
256259

257-
If your PR introduces new modules that should *not* be documented in the [API reference](target-api), or if there are changes to existing modules that necessitate their removal from the documentation, make sure to update the `exclude_modules` list within the `docs/make_api_index.py` script to reflect these exclusions.
260+
If your PR introduces new modules that should *not* be documented in the [API reference](target-api), or if there are changes to existing modules that necessitate their removal from the documentation, make sure to update `EXCLUDE_MODULES` in `docs/make_api.py` accordingly.
261+
262+
Likewise, if you want to document a module that exposes its public API via its `__init__.py`, rather than through its submodules individually, make sure to add it to `PACKAGE_MODULES` in `docs/make_api.py`.
258263

259264
### Updating the examples
260265
We use [sphinx-gallery](sphinx-gallery:)

docs/Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ help:
1515

1616
.PHONY: help Makefile
1717

18-
# Generate the API index file
18+
# Generate the API documentation
1919
api_index.rst:
20-
python make_api_index.py
20+
python make_api.py
2121

2222
# Generate the snippets/admonitions.md file
2323
# by converting the admonitions in the repo's README.md to MyST format

docs/make.bat

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,8 +35,8 @@ if "%1" == "clean" (
3535
rmdir /S /Q %SOURCEDIR%\examples\
3636
del /Q %SOURCEDIR%\snippets\admonitions.md
3737
) else (
38-
echo Generating API index...
39-
python make_api_index.py
38+
echo Generating API documentation...
39+
python make_api.py
4040

4141
echo Converting admonitions...
4242
python convert_admonitions.py

docs/make_api.py

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
"""Generate the API index and autosummary pages for ``movement`` modules.
2+
3+
This script generates the top-level API index file (``api_index.rst``)
4+
for all modules in the `movement` package, except for those specified
5+
in ``EXCLUDE_MODULES``.
6+
This script also allows "package modules" that aggregate submodules
7+
via their ``__init__.py`` files (e.g. ``movement.kinematics``) to be added
8+
to the API index, rather than listing each submodule separately.
9+
These modules are specified in ``PACKAGE_MODULES`` and will have their
10+
autosummary pages generated.
11+
"""
12+
13+
import importlib
14+
import inspect
15+
import os
16+
import sys
17+
from pathlib import Path
18+
19+
from jinja2 import FileSystemLoader
20+
from jinja2.sandbox import SandboxedEnvironment
21+
from sphinx.ext.autosummary.generate import _underline
22+
from sphinx.util import rst
23+
24+
# Single-file modules to exclude from the API index
25+
EXCLUDE_MODULES = {
26+
"movement.cli_entrypoint",
27+
"movement.napari.loader_widgets",
28+
"movement.napari.meta_widget",
29+
}
30+
31+
# Modules with __init__.py that expose submodules explicitly
32+
PACKAGE_MODULES = {"movement.kinematics", "movement.plots", "movement.roi"}
33+
34+
# Configure paths
35+
SCRIPT_DIR = Path(__file__).resolve().parent
36+
MOVEMENT_ROOT = SCRIPT_DIR.parent
37+
SOURCE_PATH = Path("source")
38+
TEMPLATES_PATH = SOURCE_PATH / "_templates"
39+
40+
os.chdir(SCRIPT_DIR)
41+
sys.path.insert(0, str(MOVEMENT_ROOT))
42+
43+
44+
def get_modules():
45+
"""Return all modules to be documented."""
46+
# Gather all modules and their paths
47+
module_names = set()
48+
for path in sorted((MOVEMENT_ROOT / "movement").rglob("*.py")):
49+
module_name = str(
50+
path.relative_to(MOVEMENT_ROOT).with_suffix("")
51+
).replace(os.sep, ".")
52+
if path.name == "__init__.py":
53+
parent = module_name.rsplit(".", 1)[0]
54+
if parent in PACKAGE_MODULES:
55+
module_names.add(parent)
56+
else:
57+
module_names.add(module_name)
58+
# Determine submodules of package modules to exclude
59+
PACKAGE_MODULE_CHILDREN = {
60+
name
61+
for name in module_names
62+
for parent in PACKAGE_MODULES
63+
if name.startswith(parent + ".")
64+
}
65+
return module_names - EXCLUDE_MODULES - PACKAGE_MODULE_CHILDREN
66+
67+
68+
def get_members(module_name):
69+
"""Return all functions and classes in a module."""
70+
mod = importlib.import_module(module_name)
71+
functions = []
72+
classes = []
73+
for name in getattr(mod, "__all__", dir(mod)):
74+
obj = getattr(mod, name, None)
75+
if inspect.isfunction(obj):
76+
functions.append(f"{name}")
77+
elif inspect.isclass(obj):
78+
classes.append(f"{name}")
79+
return sorted(functions), sorted(classes)
80+
81+
82+
def write_autosummary_module_page(module_name, output_path):
83+
"""Generate an .rst file with autosummary listing for the given module."""
84+
functions, classes = get_members(module_name)
85+
env = SandboxedEnvironment(loader=FileSystemLoader(TEMPLATES_PATH))
86+
# Add custom autosummary filters
87+
env.filters["escape"] = rst.escape
88+
env.filters["underline"] = _underline
89+
template = env.get_template("autosummary/module.rst")
90+
content = template.render(
91+
fullname=module_name,
92+
underline="=" * len(module_name),
93+
classes=classes,
94+
functions=functions,
95+
)
96+
output_path.parent.mkdir(parents=True, exist_ok=True)
97+
output_path.write_text(content)
98+
99+
100+
def make_api_index(module_names):
101+
"""Create a top-level API index file listing the specified modules."""
102+
doctree_lines = [
103+
f" {module_name}" for module_name in sorted(module_names)
104+
]
105+
api_head = (TEMPLATES_PATH / "api_index_head.rst").read_text()
106+
output_path = SOURCE_PATH / "api_index.rst"
107+
output_path.write_text(api_head + "\n" + "\n".join(doctree_lines))
108+
109+
110+
if __name__ == "__main__":
111+
# Generate autosummary pages for manual modules
112+
for module_name in PACKAGE_MODULES:
113+
output_path = SOURCE_PATH / "api" / f"{module_name}.rst"
114+
write_autosummary_module_page(module_name, output_path)
115+
# Generate the API index
116+
make_api_index(get_modules())

docs/make_api_index.py

Lines changed: 0 additions & 41 deletions
This file was deleted.

docs/source/blog/arc-collaboration.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -49,12 +49,12 @@ It is early days currently, but you can join in the conversation for what's plan
4949

5050
### Plotting Made Easier
5151

52-
The `movement.plots` submodule has been created, which provides some helpful wrapper functions for producing some of the more common plot types that come out of the analysis of `movement` datasets.
53-
These plots can be added to existing figure axes you have created, and you can pass them the same formatting arguments as you would to the appropriate `matplotlib.pyplot`.
52+
The {mod}`movement.plots` submodule has been created, which provides some helpful wrapper functions for producing some of the more common plot types that come out of the analysis of `movement` datasets.
53+
These plots can be added to existing figure axes you have created, and you can pass them the same formatting arguments as you would to the appropriate {mod}`matplotlib.pyplot`.
5454
Currently, the submodule has two wrappers to use:
5555

56-
- `plot_centroid_trajectory`, which creates a plot of the trajectory of a given keypoint (or the centroid of a collection of keypoints) with a single line of code. Introduced in [#394](movement-github:pull/394).
57-
- `plot_occupancy`, which creates an occupancy plot of an individual, given its position data. Collections of individuals are aggregated over, and if multiple keypoints are provided, the occupancy of the centroid is calculated. Introduced in [#403](movement-github:pull/403).
56+
- {func}`plot_centroid_trajectory<movement.plots.plot_centroid_trajectory>`, which creates a plot of the trajectory of a given keypoint (or the centroid of a collection of keypoints) with a single line of code. Introduced in [#394](movement-github:pull/394).
57+
- {func}`plot_occupancy<movement.plots.plot_occupancy>`, which creates an occupancy plot of an individual, given its position data. Collections of individuals are aggregated over, and if multiple keypoints are provided, the occupancy of the centroid is calculated. Introduced in [#403](movement-github:pull/403).
5858

5959
Examples to showcase the use of these plotting functions are currently [being produced](movement-github:issues/415).
6060
[Our other examples](target-examples) have also been updated to use these functions where possible.
@@ -73,20 +73,20 @@ You can find a new {ref}`example <sphx_glr_examples_advanced_broadcasting_your_o
7373

7474
One of the biggest feature introductions to `movement` during this period is support for RoIs.
7575
We now support one-dimensional regions of interest (segments, or piecewise-linear structures formed of multiple segments) as well as two-dimensional polygonal regions.
76-
You can create RoIs using the {class}`LineOfInterest <movement.roi.line.LineOfInterest>` and {class}`PolygonOfInterest <movement.roi.polygon.PolygonOfInterest>` classes respectively.
77-
There are more details about how the class is implemented on top of [`shapely`](https://shapely.readthedocs.io/en/stable/) in the {class}`docstring <movement.roi.base.BaseRegionOfInterest>`, and a note about how the interior of 2D shapes is handled.
76+
You can create RoIs using the {class}`LineOfInterest <movement.roi.LineOfInterest>` and {class}`PolygonOfInterest <movement.roi.PolygonOfInterest>` classes respectively.
77+
There are more details about how the base class is implemented on top of [`shapely`](https://shapely.readthedocs.io/en/stable/) in the {class}`docstring <movement.roi.BaseRegionOfInterest>`, and a note about how the interior of 2D shapes is handled.
7878

7979
RoIs support the calculation of certain quantities from experimental data.
8080
Highlights include:
8181

82-
- Determining if a given point(s) is inside the RoI, {func}`contains_point <movement.roi.base.BaseRegionOfInterest.contains_point>`.
83-
- Determining the distance from a given point(s) to the closest point on the RoI, {func}`compute_distance_to <movement.roi.base.BaseRegionOfInterest.compute_distance_to>`.
84-
- Determining the approach vector from a given point(s) to the RoI, {func}`compute_approach_vector <movement.roi.base.BaseRegionOfInterest.compute_approach_vector>`.
85-
- Determining egocentric and allocentric boundary angles, relative to a given RoI. See {func}`compute_allocentric_angle_to_nearest_point <movement.roi.base.BaseRegionOfInterest.compute_allocentric_angle_to_nearest_point>` and {func}`compute_egocentric_angle_to_nearest_point <movement.roi.base.BaseRegionOfInterest.compute_egocentric_angle_to_nearest_point>` for more information.
82+
- Determining if a given point(s) is inside the RoI, {func}`contains_point()<movement.roi.BaseRegionOfInterest.contains_point>`.
83+
- Determining the distance from a given point(s) to the closest point on the RoI, {func}`compute_distance_to()<movement.roi.BaseRegionOfInterest.compute_distance_to>`.
84+
- Determining the approach vector from a given point(s) to the RoI, {func}`compute_approach_vector()<movement.roi.BaseRegionOfInterest.compute_approach_vector>`.
85+
- Determining egocentric and allocentric boundary angles, relative to a given RoI. See {func}`compute_allocentric_angle_to_nearest_point()<movement.roi.BaseRegionOfInterest.compute_allocentric_angle_to_nearest_point>` and {func}`compute_egocentric_angle_to_nearest_point()<movement.roi.BaseRegionOfInterest.compute_egocentric_angle_to_nearest_point>` for more information.
8686

8787
### What's next?
8888

8989
We have [an example underway](movement-github:issues/415) that will demonstrate how to load in a dataset, define a region of interest, and query the loaded trajectories for the time periods when they were inside or outside the defined region.
9090
We also have [another example in the works](movement-github:pull/440) that will go through the boundary-angle methods.
9191

92-
Looking forward, [we will be aiming to](movement-github:issues/378) extend the `napari` plugin's [capabilities](../user_guide/gui.md) to allow users to define RoIs graphically, from within the `napari` GUI, by clicking points to define regions.
92+
Looking forward, [we will be aiming to](movement-github:issues/378) extend the `napari` plugin's [capabilities](target-gui) to allow users to define RoIs graphically, from within the `napari` GUI, by clicking points to define regions.

docs/source/conf.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@
7777

7878
# Automatically generate stub pages for API
7979
autosummary_generate = True
80+
autosummary_generate_overwrite = False
8081
autodoc_default_flags = ["members", "inherited-members"]
8182

8283
# Prefix section labels with the document name

docs/source/user_guide/movement_dataset.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -261,7 +261,7 @@ ds["velocity"] = compute_velocity(ds.position)
261261
ds.velocity
262262
```
263263

264-
The output of {func}`compute_velocity<movement.kinematics.kinematics.compute_velocity>` is an {class}`xarray.DataArray` object,
264+
The output of {func}`movement.kinematics.compute_velocity` is an {class}`xarray.DataArray` object,
265265
with the same **dimensions** as the original `position` **data variable**,
266266
so adding it to the existing `ds` makes sense and works seamlessly.
267267

movement/io/nwb.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
"""Helpers to convert between movement poses datasets and NWB files.
1+
"""Helpers to convert between ``movement`` poses datasets and NWB files.
22
33
The pose tracks in NWB files are formatted according to the ``ndx-pose``
44
NWB extension, see https://github.com/rly/ndx-pose.

movement/plots/__init__.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
"""Plotting utilities for ``movement`` datasets."""
2+
13
from movement.plots.occupancy import plot_occupancy
24
from movement.plots.trajectory import plot_centroid_trajectory
35

0 commit comments

Comments
 (0)