You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This document tracks the history and evolution of the **Open Data Contract Standard**.
2
8
3
-
# v2.2.2 - 2024-01-05 - OPEN
9
+
# v2.2.2 - 2024-05-23 - APPROVED
4
10
5
-
* Change `dataset.description` data type from `array` to `string`
6
-
* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`
7
-
* Change `price.priceAmount` data type from `string` to `number`
8
-
* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`
9
-
* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`
10
-
* Update [examples](docs/examples) to adhere to JSON schema
11
-
* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml)
12
-
* Add in mkdocs for creating documentation website
11
+
* In JSON schema validation:
12
+
* Change `dataset.description` data type from `array` to `string`.
13
+
* Change `dataset.column.isPrimaryKey` data type from `string` to `boolean`.
14
+
* Change `price.priceAmount` data type from `string` to `number`.
15
+
* Change `slaProperties.value` data type from `string` to `oneOf[string, number]`.
16
+
* Change `slaProperties.valueExt` data type from `string` to `oneOf[string, number]`.
17
+
* Update [examples](docs/examples) to adhere to JSON schema.
18
+
* Full example from README directs to [full-example.yaml](docs/examples/all/full-example.yaml).
19
+
* Add in mkdocs for creating a [documentation website](https://bitol-io.github.io/open-data-contract-standard/). Check [building-doc.md](building-doc.md).
* Type of definition for authority have standard values: `businessDefinition`, `transformationImplementation`, `videoTutorial`, `tutorial`, and `implementation`.
18
26
* Add in `isUnique`, `primaryKeyPosition`, `partitionKeyPosition`, and `clusterKeyPosition` to `column` definition.
19
-
* Add [JSON schema](schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
27
+
* Add [JSON schema](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json) to validate YAML files for v2.2.1.
20
28
* Integrated as part of [Bitol](https://lfaidata.foundation/projects/bitol/).
[](https://www.bestpractices.dev/projects/8149)
2
8
3
9
# Open Data Contract Standard (ODCS)
@@ -7,29 +13,32 @@ Welcome!
7
13
Thanks for your interest and for taking the time to come here! ❤️
8
14
9
15
## Executive summary
10
-
This standard describes a structure for a **data contract**. It's current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
16
+
This standard describes a structure for a **data contract**. Its current version is v2.2.2. It is available for you as an Apache 2.0 license. Contributions are welcome!
11
17
12
18
## Discover the open standard
13
-
Discover the [Open Data Contract Standard](docs/index.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
19
+
A reader-friendly version of the standard can be found on its [dedicated site](https://bitol-io.github.io/open-data-contract-standard/).
20
+
21
+
Discover the [Open Data Contract Standard](docs/standard.md). This file contains some explanations and several examples. More [examples](docs/examples/index.md) can be found here.
14
22
15
23
## What is a Data Contract?
16
24
17
25
### The basics of a data contract
18
26
A data contract defines the agreement between a data producer and consumers. A data contract contains several sections:
*Figure 1: illustration of a data contract, its principal contributors, sections, and usage.*
29
38
30
39
### JSON Schema
31
40
32
-
JSON Schema for ODCS can be found [here](schema/odcs-json-schema.json). You can import this schema into your IDE for
41
+
JSON Schema for ODCS can be found [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/schema/odcs-json-schema.json). You can import this schema into your IDE for
33
42
validation of your YAML files. Links below show how you can import the schema:
@@ -38,19 +47,23 @@ validation of your YAML files. Links below show how you can import the schema:
38
47
## Contributing to the project
39
48
Check out the [CONTRIBUTING](./CONTRIBUTING.md) file.
40
49
41
-
## Articles
42
-
* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879)
43
-
* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148)
44
-
* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
45
-
* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
46
-
* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d)
* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/)
49
-
* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/)
50
-
* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130)
51
-
* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1)
52
-
* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3)
53
-
* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/)
* 2024-05-25 - [Conceptual model of Data Quality of Service as Code by Jarkko Moilanen](https://aidausergroup.org/2024/05/25/aida-user-group-forecaster-pi-day-highlights-data-quality-whats-new/)
54
+
* 2024-02-06 - [Getting started with ODCS](https://medium.com/abeadata/getting-started-with-odcs-3ba790707879)
55
+
* 2023-12-08 - [Why the Need for Standardizing Data Contracts?](https://medium.com/abeadata/why-the-need-for-standardizing-data-contracts-133bc3491148)
56
+
* 2023-11-30 - [Linux Foundation AI & Data - Bitol Joins LF AI & Data as New Sandbox Project](https://lfaidata.foundation/blog/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
57
+
* 2023-11-30 - [AIDAUG - Bitol Joins LF AI & Data as New Sandbox Project](https://aidausergroup.org/2023/11/30/bitol-joins-lf-ai-data-as-new-sandbox-project/)
58
+
* 2023-11-22 - [What is, and what isn’t, a data contract](https://datacreation.substack.com/p/what-is-and-what-isnt-a-data-contract)
59
+
* 2023-10-01 - [Data Contracts: A Bridge Connecting Two Worlds](https://medium.com/@atanas.iliev.ai/data-contracts-a-bridge-connecting-two-worlds-404eff1d970d)
* 2023-08-10 - [Welcome to the Open Data Contract Standard](https://jgp.ai/2023/08/09/welcome-to-the-open-data-contract-standard/)
62
+
* 2023-05-11 - [Data Contracts – Everything You Need to Know](https://www.montecarlodata.com/blog-data-contracts-explained/)
63
+
* 2023-05-07 - [Data Engineering Weekly #130 - Data Contract in the Wild with PayPal’s Data Contract Template](https://www.dataengineeringweekly.com/p/data-engineering-weekly-130)
64
+
* 2023-05-06 - [PayPal เปิด Data Contract เป็น Open Source Template ให้ไปใช้งานกัน](https://discuss.dataengineercafe.io/t/paypal-data-contract-open-source-template/581/1)
65
+
* 2023-05-05 - [Jonathan Neo (j__neo ) on Reddit](https://www.reddit.com/r/dataengineering/comments/137glbo/comment/jixw5hj/?utm_source=reddit&utm_medium=web2x&context=3)
66
+
* 2023-05-01 - [PayPal open sources its data contract template](https://jgp.ai/2023/05/01/paypal-open-sources-its-data-contract-template/)
54
67
55
68
If you spot an article about the Open Data Contract Standard, make a pull request!
56
69
@@ -62,27 +75,4 @@ Formerly known as the data contract template, this standard is used to implement
62
75
### How does PayPal use Data Contracts?
63
76
PayPal uses data contracts in many ways, but this [article](https://medium.com/paypal-tech/the-next-generation-of-data-platforms-is-the-data-mesh-b7df4b825522) from the [PayPal Technology blog](https://medium.com/paypal-tech) gives a good introduction.
64
77
65
-
### Mkdocs mike versioning
66
-
To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
67
-
68
-
```bash
69
-
pip install mike
70
-
cd open-data-contract-standard #ensure you are inside the repo
71
-
mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
72
-
mike set-default --push latest #by default, users will go to latest
73
-
```
74
-
75
-
#### Deploying a new version
76
-
Given that the Github action [here](.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
77
-
created, all that is required is to:
78
-
1.[Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
79
-
2. Put in new tag version for release (follows pattern v*)
80
-
3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
81
-
82
-
#### Delete version
83
-
If a version tag was pushed that was incorrect, it can be deleted via:
84
-
85
-
```bash
86
-
mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
Starting with ODCS v2.2.2, the documentation is available through [GitHub.io](https://bitol-io.github.io/open-data-contract-standard). This page explains the process of building the doc.
3
+
4
+
## Mkdocs mike versioning
5
+
To start with using [mike](https://github.com/jimporter/mike) as a tool for versioning the documentation, the following was run:
6
+
7
+
```bash
8
+
pip install mike
9
+
cd open-data-contract-standard #ensure you are inside the repo
10
+
mike deploy --push --update-aliases v2.2.1 latest #set latest version to v2.2.1
11
+
mike set-default --push latest #by default, users will go to latest
12
+
```
13
+
14
+
## Deploying a new version
15
+
Given that the Github action [here](https://github.com/bitol-io/open-data-contract-standard/blob/main/.github/workflows/docs-site-deploy.yaml) it set to trigger when a new tag version is
16
+
created, all that is required is to:
17
+
1.[Create a new release](https://github.com/bitol-io/open-data-contract-standard/releases)
18
+
2. Put in new tag version for release (follows pattern v*)
19
+
3. Once release is created with new tag version, the Github action gets kicked off and mike will deploy the latest documentation linked to latest version tag
20
+
21
+
## Delete version
22
+
If a version tag was pushed that was incorrect, it can be deleted via:
23
+
24
+
```bash
25
+
mike deploy --push --update-aliases <previous tag version> latest #set latest version to previous tag version
0 commit comments