refactor output schema support to provide more than XML output (#128)

Author: tomkralidis

README.md

@@ -37,6 +37,12 @@ python setup.py install
 ### From the command line
 
 ```bash
+# show all subcommands
+pygeometa
+
+# show all supported schemas
+pygeometa schemas
+
 # provide a basic sanity check/report on an MCF
 pygeometa info --mcf=path/to/file.yml
 
@@ -64,21 +70,26 @@ Schemas supported by pygeometa:
 ### Using the API from Python
 
 ```python
-from pygeometa.core import render_template
+from pygeometa.core import read_mcf, render_j2_template
+
+# read from disk
+mcf_dict = read_mcf('/path/to/file.yml')
+# read from string
+mcf_dict = read_mcf(mcf_string)
+
+# choose ISO 19139 output schema
+from pygeometa.schemas.iso19139 import ISO19139OutputSchema
+iso_os = ISO19139OutputSchema()
+
 # default schema
-xml_string = render_template('/path/to/file.yml', schema='iso19139')
+xml_string = render_j2_template(mcf_dict, template_dir=iso_os.template_dir)
+
 # user-defined schema
-xml_string = render_template('/path/to/file.yml', schema_local='/path/to/new-schema')
-# dictionary representation of YAML
-xml_string = render_template(yaml_dict, schema='iso19139')
-with open('output.xml', 'w') as ff:
+xml_string = render_j2_template(mcf_dict, template_dir='/path/to/new-schema')
+
+# write to disk
+with open('output.xml', 'wb') as ff:
     ff.write(xml_string)
-# render from an MCF stored in a string
-mcf_string = '...'  # some string
-xml_string = render_template_string(mcf_string, schema='iso19139')
-# render from an MCF as a ConfigParser object
-mcf_cp = '...'  # some ConfigParser object
-xml_string = render_template_string(mcf_cp, schema='iso19139')
 ```
 
 ## Migration
@@ -107,15 +118,40 @@ pip install -r requirements-dev.txt
 
 ### Adding a Metadata Schema to the Core
 
-List of supported metadata schemas in `pygeometa/templates/`
+Adding an output metadata schemas to pygeometa involves extending
+`pygeometa.schemas.base.BaseOutputSchema` and supporting the `write`
+function to return a string of exported metadata content.  If you are using
+Jinja2 templates, see the next section.  If you are using another means of
+generating metadata (lxml, xml.etree, json, etc.), override the ABS `write`
+class to emit a string using your tooling/workflow accordingly.  See the
+below sections for examples.
+
+Once you have added your metadata schema, you need to register it with
+pygeometa's schema registry:
 
-To add support to new metadata schemas:
 ```bash
-cp -r pygeometa/templates/iso19139 pygeometa/templates/new-schema
+vi pygeometa/schemas/__init__.py
+# edit the SCHEMAS dict with the metadata schema name and dotted path of class
 ```
-Then modify `*.j2` files in the new `pygeometa/templates/new-schema` directory
+
+#### Jinja2 templates
+
+To add support for a new metadata schema using Jinja2 templates:
+```bash
+cp -r pygeometa/schemas/iso19139 pygeometa/schemas/new-schema
+```
+Then modify `*.j2` files in the new `pygeometa/schemas/new-schema` directory
 to comply to new metadata schema.
 
+#### Custom tooling
+
+To add support for a new metadata schemas using other tooling/workflow:
+```bash
+mkdir pygeometa/schemas/foo
+cp pygeometa/schemas/iso19139/__init__.py pygeometa/schemas/foo
+vi pygeometa/schemas/foo/__init__.py
+# update class name and super().__init__() function accordingly 
+
 ### Running Tests
 
 ```bash

docs/content/index.md

@@ -23,13 +23,13 @@ Feel free to open an [issue](https://github.com/geopython/pygeometa/issues)!
 pygeometa has minimal dependencies (install is less than 50K), and provides
 a flexible extension mechanism leveraging the Jinja2 templating system.
 
-pygeoapi is [open source](https://opensource.org) and released under an
-[MIT license](https://github.com/geopython/pygeoapi/blob/master/LICENSE.md).
+pygeometa is [open source](https://opensource.org) and released under an
+[MIT license](https://github.com/geopython/pygeometa/blob/master/LICENSE.md).
 
 ## Features
 * simple YAML-based configuration
-* extensible: template architecture allows for easy addition of new metadata
-  formats
+* extensible: plugin architecture allows for easy addition of new metadata
+  formats using Jinja2 templates or custom workflow (JSON, YAML, CSV, etc.)
 * flexible: use as a command-line tool or integrate as a library
 
 ## History

docs/content/reference/formats/hnap.md renamed to docs/content/reference/formats/iso19139-hnap.md

@@ -3,6 +3,6 @@
 * [Metadata Control File](mcf)
 * Formats
     * iso19139, [specification](http://www.iso.org/iso/catalogue_detail.htm?csnumber=32557)
-    * [iso19139-hnap](formats/hnap), [specification](http://www.gcpedia.gc.ca/wiki/Federal_Geospatial_Platform/Policies_and_Standards/Catalogue/Release/Appendix_B_Guidelines_and_Best_Practices/Guide_to_Harmonized_ISO_19115:2003_NAP)
+    * [iso19139-hnap](formats/iso19139-hnap), [specification](http://www.gcpedia.gc.ca/wiki/Federal_Geospatial_Platform/Policies_and_Standards/Catalogue/Release/Appendix_B_Guidelines_and_Best_Practices/Guide_to_Harmonized_ISO_19115:2003_NAP)
     * [wmo-cmp](formats/wmo-cmp), [specification](http://wis.wmo.int/2013/metadata/version_1-3-0/WMO_Core_Metadata_Profile_v1.3_Part_1.pdf) 
     * [wmo-wigos](formats/wmo-wigos.md), [specification](https://library.wmo.int/opac/doc_num.php?explnum_id=3653)

docs/content/reference/index.md

@@ -35,6 +35,12 @@ The basic pygeometa workflow is:
 ### Running
 
 ```bash
+# show all subcommands
+pygeometa
+
+# show all supported schemas
+pygeometa schemas
+
 # provide a basic sanity check/report on an MCF (Metadata Control File)
 pygeometa info --mcf=path/to/file.yml
 
@@ -95,21 +101,26 @@ python setup.py install
 ### Using the API from Python
 
 ```python
-from pygeometa.core import render_template
+from pygeometa.core import read_mcf, render_j2_template
+
+# read from disk
+mcf_dict = read_mcf('/path/to/file.yml')
+# read from string
+mcf_dict = read_mcf(mcf_string)
+
+# choose ISO 19139 output schema
+from pygeometa.schemas.iso19139 import ISO19139OutputSchema
+iso_os = ISO19139OutputSchema()
+
 # default schema
-xml_string = render_template('/path/to/file.yml', schema='iso19139')
+xml_string = render_j2_template(mcf_dict, template_dir=iso_os.template_dir)
+
 # user-defined schema
-xml_string = render_template('/path/to/file.yml', schema_local='/path/to/new-schema')
-# dictionary representation of YAML
-xml_string = render_template(yaml_dict, schema='iso19139')
-with open('output.xml', 'w') as ff:
+xml_string = render_j2_template(mcf_dict, schema_local='/path/to/new-schema')
+
+# write to disk
+with open('output.xml', 'wb') as ff:
     ff.write(xml_string)
-# render from an MCF stored in a string
-mcf_string = '...'  # some string
-xml_string = render_template_string(mcf_string, schema='iso19139')
-# render from an MCF as a ConfigParser object
-mcf_cp = '...'  # some ConfigParser object
-xml_string = render_template_string(mcf_cp, schema='iso19139')
 ```
 
 ## Development
@@ -125,15 +136,41 @@ pip install -r requirements-dev.txt
 
 ### Adding a Metadata Schema to the Core
 
-List of supported metadata schemas in `pygeometa/templates/`
+Adding an output metadata schemas to pygeometa involves extending
+`pygeometa.schemas.base.BaseOutputSchema` and supporting the `write`
+function to return a string of exported metadata content.  If you are using
+Jinja2 templates, see the next section.  If you are using another means of
+generating metadata (lxml, xml.etree, json, etc.), override the ABS `write`
+class to emit a string using your tooling/workflow accordingly.  See the
+below sections for examples.
+
+Once you have added your metadata schema, you need to register it with
+pygeometa's schema registry:
 
-To add support to new metadata schemas:
 ```bash
-cp -r pygeometa/templates/iso19139 pygeometa/templates/new-schema
+vi pygeometa/schemas/__init__.py
+# edit the SCHEMAS dict with the metadata schema name and dotted path of class
 ```
-Then modify `*.j2` files in the new `pygeometa/templates/new-schema` directory
+
+#### Jinja2 templates
+
+To add support for a new metadata schema using Jinja2 templates:
+```bash
+cp -r pygeometa/schemas/iso19139 pygeometa/schemas/new-schema
+```
+Then modify `*.j2` files in the new `pygeometa/schemas/new-schema` directory
 to comply to new metadata schema.
 
+#### Custom tooling
+
+To add support for a new metadata schemas using other tooling/workflow:
+```bash
+mkdir pygeometa/schemas/foo
+cp pygeometa/schemas/iso19139/__init__.py pygeometa/schemas/foo
+vi pygeometa/schemas/foo/__init__.py
+# update class name and super().__init__() function accordingly 
+```
+
 ### Running Tests
 
 ```bash
@@ -169,4 +206,3 @@ metadata within a real-time environment and automated workflows.
 
 In 2015 pygeometa was made publically available in support of the Treasury
 Board [Policy on Acceptable Network and Device Use](http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=27122).
-

docs/content/tutorial.md

@@ -13,7 +13,7 @@ nav:
     - Reference:
         - Metadata Control File: reference/mcf.md
         - Formats:
-            - ISO Harmonized North American Profile (HNAP): reference/formats/hnap.md
+            - ISO Harmonized North American Profile (HNAP): reference/formats/iso19139-hnap.md
             - WMO Core Metadata Profile: reference/formats/wmo-cmp.md
             - WMO WIGOS Metadata Standard: reference/formats/wmo-wigos.md
 #        - Index: reference/index.md

docs/mkdocs.yml

@@ -46,7 +46,7 @@
 
 import click
 
-from pygeometa.core import generate_metadata, info
+from pygeometa.core import generate_metadata, info, schemas
 from pygeometa.migrations import migrate
 
 __version__ = '0.6.dev0'
@@ -60,4 +60,5 @@ def cli():
 
 cli.add_command(generate_metadata)
 cli.add_command(info)
+cli.add_command(schemas)
 cli.add_command(migrate)