Skip to content

Commit 0f63de4

Browse files
committed
Merge remote-tracking branch 'bitol/main' into dev
# Conflicts: # site/404.html # site/contributing/index.html # site/examples/all/full-example.yaml # site/examples/index.html # site/examples/quality/column-completeness.yaml # site/examples/quality/column-validity.yaml # site/index.html # site/search/search_index.json # site/sitemap.xml # site/sitemap.xml.gz # site/standard/index.html
2 parents 5e9441a + 38ecbc4 commit 0f63de4

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+129
-13803
lines changed

.github/workflows/docs-site-deploy.yaml

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
name: docs-site-deploy
22
on:
33
push:
4-
tags:
5-
- "v*"
4+
branches:
5+
- main
66
permissions:
77
contents: write
88
jobs:
@@ -28,5 +28,8 @@ jobs:
2828
mkdocs-material-
2929
- run: pip install mkdocs-open-in-new-tab mike
3030
- run: pip install mkdocs-material
31-
- run: mkdocs build
32-
- run: mike deploy --push --update-aliases ${{ github.ref_name }} latest
31+
- run: pip install "mkdocs-material[imaging]"
32+
- run: bash script/build_docs.sh
33+
- run: |
34+
latest_tag=$(git describe --tags --abbrev=0)
35+
mike deploy --push --update-aliases "$latest_tag" latest

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
.idea
22
.DS_Store
33
.idea
4+
site
5+
*.iml

CHANGELOG.md

Lines changed: 18 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,30 @@
1+
---
2+
title: "Changelog: Open Data Contract Standard (ODCS)"
3+
description: "Home of Open Data Contract Standard (ODCS) documentation."
4+
image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color/Bitol_Logo_color.svg"
5+
---
6+
17
This document tracks the history and evolution of the **Open Data Contract Standard**.
28

3-
# v2.2.2 - 2024-01-05 - OPEN
9+
# v2.2.2 - 2024-05-23 - APPROVED
410

5-
* Change `dataset.description` data type from `array` to `string`
6-
* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`
7-
* Change `price.priceAmount` data type from `string` to `number`
8-
* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`
9-
* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`
10-
* Update [examples](docs/examples) to adhere to JSON schema
11-
* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml)
12-
* Add in mkdocs for creating documentation website
11+
* In JSON schema validation:
12+
* Change `dataset.description` data type from `array` to `string`.
13+
* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`.
14+
* Change `price.priceAmount` data type from `string` to `number`.
15+
* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`.
16+
* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`.
17+
* Update [examples](docs/examples) to adhere to JSON schema.
18+
* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml).
19+
* Add in mkdocs for creating a [documentation website](https://bitol-io.github.io/open-data-contract-standard/). Check [building-doc.md](building-doc.md).
20+
* Add vendors page [vendors.md](vendors.md). Feel free to add anyone there.
1321

1422
# v2.2.1 - 2023-12-18 - APPROVED
1523

1624
* Reformat quality examples to be valid YAML.
1725
* Type of definition for authority have standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.
1826
* Add in `isUnique`, `primaryKeyPosition`, `partitionKeyPosition`, and `clusterKeyPosition` to `column` definition.
19-
* Add [JSON schema](schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
27+
* Add [JSON schema](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
2028
* Integrated as part of [Bitol](https://lfaidata.foundation/projects/bitol/).
2129
* Reformat Markdown tables.
2230

CONTRIBUTING.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
---
2+
title: "Contributing: Open Data Contract Standard (ODCS)"
3+
description: "How you can contribute to the Open Data Contract Standard (ODCS)."
4+
image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color/Bitol_Logo_color.svg"
5+
---
6+
17
# Open Data Contract Standard
28

39
## Executive summary

README.md

Lines changed: 37 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
---
2+
title: "Open Data Contract Standard (ODCS)"
3+
description: "Home of Open Data Contract Standard (ODCS) documentation."
4+
image: "https://raw.githubusercontent.com/bitol-io/artwork/main/horizontal/color/Bitol_Logo_color.svg"
5+
---
6+
17
[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/8149/badge)](https://www.bestpractices.dev/projects/8149)
28

39
# Open Data Contract Standard (ODCS)
@@ -7,29 +13,32 @@ Welcome!
713
Thanks for your interest and for taking the time to come here! ❤️
814

915
## Executive summary
10-
This standard describes a structure for a **data contract**. It's current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
16+
This standard describes a structure for a **data contract**. Its current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
1117

1218
## Discover the open standard
13-
Discover the [Open Data Contract Standard](docs/index.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
19+
A reader-friendly version of the standard can be found on its [dedicated site](https://bitol-io.github.io/open-data-contract-standard/).
20+
21+
Discover the [Open Data Contract Standard](docs/standard.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
1422

1523
## What is a Data Contract?
1624

1725
### The basics of a data contract
1826
A data contract defines the agreement between a data producer and consumers. A data contract contains several sections:
19-
* Fundamentals.
20-
* Schema.
21-
* Data quality.
22-
* Service-level agreement (SLA).
23-
* Security & stakeholders.
24-
* Custom properties.
2527

26-
![Data contract schema](./docs/img/data-contract-v2.2.1-schema.svg "Data contract schema")
28+
* [Fundamentals](docs/standard.md#demographics).
29+
* [Schema](docs/standard.md#dataset-and-schema).
30+
* [Data quality](docs/standard.md#data-quality).
31+
* [Service-level agreement (SLA)](docs/standard.md#service-level-agreement).
32+
* [Security & stakeholders](docs/standard.md#stakeholders).
33+
* [Custom properties](docs/standard.md#other-properties).
34+
35+
![Data contract schema](docs/img/data-contract-v2.2.1-schema.svg "Data contract schema")
2736

2837
*Figure 1: illustration of a data contract, its principal contributors, sections, and usage.*
2938

3039
### JSON Schema
3140

32-
JSON Schema for ODCS can be found [here](schema/odcs-json-schema.json). You can import this schema into your IDE for
41+
JSON Schema for ODCS can be found [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json). You can import this schema into your IDE for
3342
validation of your YAML files. Links below show how you can import the schema:
3443

3544
- [IntelliJ](https://www.jetbrains.com/help/idea/json.html#ws_json_schema_add_custom)
@@ -38,19 +47,23 @@ validation of your YAML files. Links below show how you can import the schema:
3847
## Contributing to the project
3948
Check out the [CONTRIBUTING](./CONTRIBUTING.md) file.
4049

41-
## Articles
42-
* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879)
43-
* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148)
44-
* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
45-
* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
46-
* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d)
47-
* 2023-09-10 - [Data Contracts 101](https://medium.com/p/568a9adbf9a9)
48-
* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/)
49-
* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/)
50-
* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130)
51-
* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1)
52-
* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3)
53-
* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/)
50+
## Articles and Other Resources
51+
52+
* 2024-05-30 - [ODCS Roadmap](https://medium.com/abeadata/odcs-roadmap-9b9a17367af4)
53+
* 2024-05-25 - [Conceptual model of Data Quality of Service as Code by Jarkko Moilanen](https://aidausergroup.org/2024/05/25/aida-user-group-forecaster-pi-day-highlights-data-quality-whats-new/)
54+
* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879)
55+
* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148)
56+
* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
57+
* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
58+
* 2023-11-22 - [What is, and what isn’t, a data contract](https://datacreation.substack.com/p/what-is-and-what-isnt-a-data-contract)
59+
* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d)
60+
* 2023-09-10 - [Data Contracts 101](https://medium.com/p/568a9adbf9a9)
61+
* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/)
62+
* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/)
63+
* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130)
64+
* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1)
65+
* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3)
66+
* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/)
5467

5568
If you spot an article about the Open Data Contract Standard, make a pull request!
5669

@@ -62,27 +75,4 @@ Formerly known as the data contract template, this standard is used to implement
6275
### How does PayPal use Data Contracts?
6376
PayPal uses data contracts in many ways, but this [article](https://medium.com/paypal-tech/the-next-generation-of-data-platforms-is-the-data-mesh-b7df4b825522) from the [PayPal Technology blog](https://medium.com/paypal-tech) gives a good introduction.
6477

65-
### Mkdocs mike versioning
66-
To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
67-
68-
```bash
69-
pip install mike
70-
cd open-data-contract-standard #ensure you are inside the repo
71-
mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
72-
mike set-default --push latest #by default, users will go to latest
73-
```
74-
75-
#### Deploying a new version
76-
Given that the Github action [here](.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
77-
created, all that is required is to:
78-
1. [Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
79-
2. Put in new tag version for release (follows pattern v*)
80-
3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
81-
82-
#### Delete version
83-
If a version tag was pushed that was incorrect, it can be deleted via:
84-
85-
```bash
86-
mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
87-
mike delete <incorrect tag> --push
88-
```
78+

building-doc.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Automatic documentation builder
2+
Starting with ODCS v2.2.2, the documentation is available through [GitHub.io](https://bitol-io.github.io/open-data-contract-standard). This page explains the process of building the doc.
3+
4+
## Mkdocs mike versioning
5+
To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
6+
7+
```bash
8+
pip install mike
9+
cd open-data-contract-standard #ensure you are inside the repo
10+
mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
11+
mike set-default --push latest #by default, users will go to latest
12+
```
13+
14+
## Deploying a new version
15+
Given that the Github action [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
16+
created, all that is required is to:
17+
1. [Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
18+
2. Put in new tag version for release (follows pattern v*)
19+
3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
20+
21+
## Delete version
22+
If a version tag was pushed that was incorrect, it can be deleted via:
23+
24+
```bash
25+
mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
26+
mike delete <incorrect tag> --push
27+
```

docs/contributing.md

Lines changed: 0 additions & 85 deletions
This file was deleted.

0 commit comments

Comments
 (0)