You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
"description": "This containerized application can be used to run workflows used for ingesting dataset metadata into a Dataverse instance. The different flows and tasks used in these workflows are created using Prefect. If you run the container locally they can be monitored and ran from the Prefect Orion UI at http://localhost:4200.\n\nMost flows start with an entry workflow that can be found in the directory entry_workflows. Here the metadata is first harvested using OAI-PMH and uploaded to S3 storage. After, the metadata is fetched from that S3 storage, and a provenance object is created for the ingested metadata. A settings dictionary constructed with DynaConf that is specific to the Data Provider is also constructed here.\n\nNext, For every dataset's metadata it runs a sub-flow to handle the actual ingestion. These flows can be found in the dataset_workflows directory. The dataset workflow uses simple tasks that make an API call to a service. These services often transform, improve or alter the metadata in some way.\n\nIn short, most ingestion workflows take the following steps:\n\n-Harvest metadata and upload it to S3 storage.\n-Fetch dataset metadata from S3 storage.\n-Create version object of all services that will be used for -ingestion.\n-For every dataset's metadata run dataset workflow.\n-Use tasks that make API calls to different services to transform the metadata.",
38
+
"contributor": [
39
+
{
40
+
"@id": "https://orcid.org/0000-0002-0079-2533",
41
+
"@type": "Person",
42
+
"affiliation": {
43
+
"@type": "Organization",
44
+
"@id": "https://ror.org/008xxew50",
45
+
"name": "VU Amsterdam"
46
+
},
47
+
"email": "a.valdestilhas@vu.nl",
48
+
"familyName": "Valdestilhas",
49
+
"givenName": "Andre"
50
+
}
51
+
],
52
+
"dateCreated": "2022-11-24",
53
+
"description": "Orchestrator for all ODISSEI-related ingestion workflows.",
54
+
"funding":
55
+
[
56
+
{
57
+
"@type": "Grant",
58
+
"name": "ODISSEI: Better Infrastructure, Better Science, Better Society (NWO Grant 184.035.014)",
59
+
"funder": {
60
+
"@type": "Organization",
61
+
"name": "NWO",
62
+
"@id": "https://www.nwo.nl"
63
+
}
64
+
},
65
+
{
66
+
"@type": "Grant",
67
+
"name": "Building a FAIR Expertise Hub for the social sciences",
0 commit comments