[epic] Catalog functionality (20% analysed)

[rufuspollock]

When I have a set of datasets (in a single repo or across multiple repos or in CKAN backend) I want a catalog to display and organize them so that people can find them together

Acceptance

Tasks

• Do analysis? What's the KISS approach?

Analysis

Where is the metadata/data stored?

• multiple datasets to the folder ...
  • The main issue is rebuilding when something changes (if you rebuild everything each time one dataset changes you start to get a lot of rebuilds and slow rebuilds ...)
• multiple repos (all within one org?) on github
• What about with a backend like CKAN?
  • Gets dynamic then??

How would this work on DataHub.io?

• Is each dataset is deployed independently ... (is that a problem?)

What's the url layout? [Ans: my sense is that DataHub is no longer a single catalog but catalogs so more the second option]

datahub.io
datahub.io/dashboard
datahub.io/{org-or-group}/{dataset}

OR

user-id-dataset-id.datahub.io

# Is it that hard to combine these via a proxy ...?

datahub.io/{org-or-group}/{dataset} => renders same as user-id-dataset-id.datahub.io

# (only issue would be relative urls ...)
# and having one app per dataset
                                   

[rufuspollock]

NB: we now have multiple working catalog examples - see https://portaljs.org/examples.

We just don't yet have a catalog "spec" for how to define this in one standard