Skip to content

Commit b02be28

Browse files
xbcsmithxbcsmith
authored
Standardize values and type of outcome (#238)
* Standardize values and type of outcome Signed-off-by: Brett Smith <bc.smith@sas.com> * Add cancel to outcome values Signed-off-by: Brett Smith <xbcsmith@gmail.com> --------- Signed-off-by: Brett Smith <bc.smith@sas.com> Signed-off-by: Brett Smith <xbcsmith@gmail.com>
1 parent f6aa31c commit b02be28

File tree

7 files changed

+40
-126
lines changed

7 files changed

+40
-126
lines changed

‎conformance/testcaserun_finished.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@
4444
"source": "/event/source/123",
4545
"type": "testCaseRun",
4646
"content": {
47-
"outcome": "pass",
47+
"outcome": "success",
4848
"environment": {
4949
"id": "dev",
5050
"source": "testkube-dev-123"

‎conformance/testsuiterun_finished.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@
4444
"source": "/event/source/123",
4545
"type": "testSuiteRun",
4646
"content": {
47-
"outcome": "fail",
47+
"outcome": "failure",
4848
"severity": "critical",
4949
"reason": "Host 123.34.23.32 not found",
5050
"environment": {

‎core.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,9 @@ track the build and release progress on a particular software artifact.
3636
| source | `URI-Reference` | See [source](spec.md#source-subject) | |
3737
| type | `String` | See [type](spec.md#type-subject) | `pipelineRun` |
3838
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` |
39-
| outcome | `String` | outcome of a finished `pipelineRun` | `success`, `error` or `failure`|
39+
| outcome | `String` | outcome of a finished `pipelineRun` | `success`, `failure`, `cancel`, or `error` |
4040
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` |
41-
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun canceled by user`, `Unit tests failed`|
41+
| errors | `String` | In case of error, canceled, or failed pipeline , provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun canceled by user`, `Unit tests failed`|
4242

4343
### `taskRun`
4444

@@ -56,9 +56,9 @@ associated, in which case it is acceptable to generate only taskRun events.
5656
| type | `String` | See [type](spec.md#type-subject) | `taskRun` |
5757
| taskName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` |
5858
| pipelineRun | `Object` ([`pipelineRun`](#pipelinerun)) | The `pipelineRun` that this `taskRun` belongs to. | `{"id": "namespace/pipelinerun-1234"}`|
59-
| outcome | `String` | outcome of a finished `taskRun` | `success`, `error` or `failure`|
59+
| outcome | `String` | outcome of a finished `taskRun` | `success`, `failure`, `cancel`, or `error` |
6060
| url | `URI` | url to the `taskRun` | `https://dashboard.org/namespace/taskrun-1234`, `https://api.cdsystem.com/namespace/taskrun-1234` |
61-
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun canceled by user`, `Unit tests failed`|
61+
| errors | `String` | In case of error, canceled, or failed pipeline , provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun canceled by user`, `Unit tests failed`|
6262

6363
## Events
6464

@@ -111,8 +111,8 @@ A pipelineRun has finished, successfully or not.
111111
| type | `String` | See [type](spec.md#type-subject) | `pipelineRun` | |
112112
| pipelineName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | |
113113
| url | `URI` | url to the `pipelineRun` | `https://dashboard.org/namespace/pipelinerun-1234`, `https://api.cdsystem.com/namespace/pipelinerun-1234` | |
114-
| outcome | `String (enum)` | outcome of a finished `pipelineRun` | `success`, `error` or `failure`| |
115-
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun canceled by user`, `Unit tests failed`| |
114+
| outcome | `String (enum)` | outcome of a finished `pipelineRun` | `success`, `failure`, `cancel`, or `error` | `success`, `failure`, `cancel`, `error` |
115+
| errors | `String` | In case of error, canceled, or failed pipeline , provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `pipelineRun canceled by user`, `Unit tests failed`| |
116116

117117
### [`taskRun Started`](conformance/taskrun_started.json)
118118

@@ -147,5 +147,5 @@ A taskRun has finished, successfully or not.
147147
| taskName | `String` | The name of the pipeline | `MyPipeline`, `Unit tests for my repo` | |
148148
| pipelineRun | `Object` ([`pipelineRun`](#pipelinerun)) | The `pipelineRun` that this `taskRun` belongs to. | `{"id": "namespace/pipelinerun-1234"}`| |
149149
| url | `URI` | url to the `taskRun` | `https://dashboard.org/namespace/taskrun-1234`, `https://api.cdsystem.com/namespace/taskrun-1234` | |
150-
| outcome | `String (enum)` | outcome of a finished `taskRun` | `success`, `error` or `failure`| |
151-
| errors | `String` | In case of error or failed pipeline, provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun canceled by user`, `Unit tests failed`| |
150+
| outcome | `String (enum)` | outcome of a finished `taskRun` | `success`, `failure`, `cancel`, or `error` | `success`, `failure`, `cancel`, `error` |
151+
| errors | `String` | In case of error, canceled, or failed pipeline , provides details about the failure | `Invalid input param 123`, `Timeout during execution`, `taskRun canceled by user`, `Unit tests failed`| |

‎schemas/pipelinerunfinished.json

Lines changed: 7 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,7 @@
1919
},
2020
"type": {
2121
"type": "string",
22-
"enum": [
23-
"dev.cdevents.pipelinerun.finished.0.3.0-draft"
24-
],
22+
"enum": ["dev.cdevents.pipelinerun.finished.0.3.0-draft"],
2523
"default": "dev.cdevents.pipelinerun.finished.0.3.0-draft"
2624
},
2725
"timestamp": {
@@ -43,13 +41,7 @@
4341
},
4442
"additionalProperties": false,
4543
"type": "object",
46-
"required": [
47-
"version",
48-
"id",
49-
"source",
50-
"type",
51-
"timestamp"
52-
]
44+
"required": ["version", "id", "source", "type", "timestamp"]
5345
},
5446
"subject": {
5547
"properties": {
@@ -65,9 +57,7 @@
6557
"type": {
6658
"type": "string",
6759
"minLength": 1,
68-
"enum": [
69-
"pipelineRun"
70-
],
60+
"enum": ["pipelineRun"],
7161
"default": "pipelineRun"
7262
},
7363
"content": {
@@ -80,7 +70,8 @@
8070
"format": "uri"
8171
},
8272
"outcome": {
83-
"type": "string"
73+
"type": "string",
74+
"enum": ["success", "failure", "cancel", "error"]
8475
},
8576
"errors": {
8677
"type": "string"
@@ -92,11 +83,7 @@
9283
},
9384
"additionalProperties": false,
9485
"type": "object",
95-
"required": [
96-
"id",
97-
"type",
98-
"content"
99-
]
86+
"required": ["id", "type", "content"]
10087
},
10188
"customData": {
10289
"oneOf": [
@@ -115,8 +102,5 @@
115102
},
116103
"additionalProperties": false,
117104
"type": "object",
118-
"required": [
119-
"context",
120-
"subject"
121-
]
105+
"required": ["context", "subject"]
122106
}

‎schemas/testcaserunfinished.json

Lines changed: 11 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,7 @@
1818
},
1919
"type": {
2020
"type": "string",
21-
"enum": [
22-
"dev.cdevents.testcaserun.finished.0.3.0-draft"
23-
],
21+
"enum": ["dev.cdevents.testcaserun.finished.0.3.0-draft"],
2422
"default": "dev.cdevents.testcaserun.finished.0.3.0-draft"
2523
},
2624
"timestamp": {
@@ -42,13 +40,7 @@
4240
},
4341
"additionalProperties": false,
4442
"type": "object",
45-
"required": [
46-
"version",
47-
"id",
48-
"source",
49-
"type",
50-
"timestamp"
51-
]
43+
"required": ["version", "id", "source", "type", "timestamp"]
5244
},
5345
"subject": {
5446
"properties": {
@@ -62,30 +54,18 @@
6254
"type": {
6355
"type": "string",
6456
"minLength": 1,
65-
"enum": [
66-
"testCaseRun"
67-
],
57+
"enum": ["testCaseRun"],
6858
"default": "testCaseRun"
6959
},
7060
"content": {
7161
"properties": {
7262
"outcome": {
7363
"type": "string",
74-
"enum": [
75-
"pass",
76-
"fail",
77-
"cancel",
78-
"error"
79-
]
64+
"enum": ["success", "failure", "cancel", "error"]
8065
},
8166
"severity": {
8267
"type": "string",
83-
"enum": [
84-
"low",
85-
"medium",
86-
"high",
87-
"critical"
88-
]
68+
"enum": ["low", "medium", "high", "critical"]
8969
},
9070
"reason": {
9171
"type": "string"
@@ -104,9 +84,7 @@
10484
},
10585
"additionalProperties": false,
10686
"type": "object",
107-
"required": [
108-
"id"
109-
]
87+
"required": ["id"]
11088
},
11189
"testSuiteRun": {
11290
"type": "object",
@@ -120,16 +98,12 @@
12098
}
12199
},
122100
"additionalProperties": false,
123-
"required": [
124-
"id"
125-
]
101+
"required": ["id"]
126102
},
127103
"testCase": {
128104
"type": "object",
129105
"additionalProperties": false,
130-
"required": [
131-
"id"
132-
],
106+
"required": ["id"],
133107
"properties": {
134108
"id": {
135109
"type": "string",
@@ -163,19 +137,12 @@
163137
},
164138
"additionalProperties": false,
165139
"type": "object",
166-
"required": [
167-
"outcome",
168-
"environment"
169-
]
140+
"required": ["outcome", "environment"]
170141
}
171142
},
172143
"additionalProperties": false,
173144
"type": "object",
174-
"required": [
175-
"id",
176-
"type",
177-
"content"
178-
]
145+
"required": ["id", "type", "content"]
179146
},
180147
"customData": {
181148
"oneOf": [
@@ -194,8 +161,5 @@
194161
},
195162
"additionalProperties": false,
196163
"type": "object",
197-
"required": [
198-
"context",
199-
"subject"
200-
]
164+
"required": ["context", "subject"]
201165
}

‎schemas/testsuiterunfinished.json

Lines changed: 10 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,7 @@
1818
},
1919
"type": {
2020
"type": "string",
21-
"enum": [
22-
"dev.cdevents.testsuiterun.finished.0.3.0-draft"
23-
],
21+
"enum": ["dev.cdevents.testsuiterun.finished.0.3.0-draft"],
2422
"default": "dev.cdevents.testsuiterun.finished.0.3.0-draft"
2523
},
2624
"timestamp": {
@@ -42,13 +40,7 @@
4240
},
4341
"additionalProperties": false,
4442
"type": "object",
45-
"required": [
46-
"version",
47-
"id",
48-
"source",
49-
"type",
50-
"timestamp"
51-
]
43+
"required": ["version", "id", "source", "type", "timestamp"]
5244
},
5345
"subject": {
5446
"properties": {
@@ -62,9 +54,7 @@
6254
"type": {
6355
"type": "string",
6456
"minLength": 1,
65-
"enum": [
66-
"testSuiteRun"
67-
],
57+
"enum": ["testSuiteRun"],
6858
"default": "testSuiteRun"
6959
},
7060
"content": {
@@ -83,16 +73,12 @@
8373
},
8474
"additionalProperties": false,
8575
"type": "object",
86-
"required": [
87-
"id"
88-
]
76+
"required": ["id"]
8977
},
9078
"testSuite": {
9179
"type": "object",
9280
"additionalProperties": false,
93-
"required": [
94-
"id"
95-
],
81+
"required": ["id"],
9682
"properties": {
9783
"id": {
9884
"type": "string",
@@ -112,41 +98,24 @@
11298
},
11399
"outcome": {
114100
"type": "string",
115-
"enum": [
116-
"pass",
117-
"fail",
118-
"cancel",
119-
"error"
120-
]
101+
"enum": ["success", "failure", "cancel", "error"]
121102
},
122103
"severity": {
123104
"type": "string",
124-
"enum": [
125-
"low",
126-
"medium",
127-
"high",
128-
"critical"
129-
]
105+
"enum": ["low", "medium", "high", "critical"]
130106
},
131107
"reason": {
132108
"type": "string"
133109
}
134110
},
135111
"additionalProperties": false,
136112
"type": "object",
137-
"required": [
138-
"outcome",
139-
"environment"
140-
]
113+
"required": ["outcome", "environment"]
141114
}
142115
},
143116
"additionalProperties": false,
144117
"type": "object",
145-
"required": [
146-
"id",
147-
"type",
148-
"content"
149-
]
118+
"required": ["id", "type", "content"]
150119
},
151120
"customData": {
152121
"oneOf": [
@@ -165,8 +134,5 @@
165134
},
166135
"additionalProperties": false,
167136
"type": "object",
168-
"required": [
169-
"context",
170-
"subject"
171-
]
137+
"required": ["context", "subject"]
172138
}

‎testing-events.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -120,7 +120,7 @@ This event represents a finished testCase execution. The event will contain the
120120
| environment | `Object` [`environment`](continuous-deployment.md/#environment) | The environment in which this testCaseRun was running | `{"id": "1234"}`, `{"id": "dev", "source": "testkube-dev-123"}` | ✅ |
121121
| testCase | `Object` [`testCase`](#testcase) | Definition of the testCase being executed | `{"id": "92834723894", "name": "Login Test", "type": "integration"}` | |
122122
| testSuiteRun | `Object` [`testSuiteRun`](#testsuiterun) | A testSuiteRun to associate this testCaseRun with a containing testSuiteRun | `{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}` | |
123-
| outcome | `String (enum)` | The outcome of the testSuite execution, one of `pass`, `fail`, `cancel`, `error` | `pass` | ✅ |
123+
| outcome | `String (enum)` | The outcome of the testSuite execution, one of `success`, `failure`, `cancel`, or `error` | `success`, `failure`, `cancel`, `error` | ✅ |
124124
| severity | `String (enum)` | Severity if the test failed, one of `low`, `medium`, `high`, `critical` | `critical` |
125125
| reason | `String` | A reason related to the outcome of the execution | `Canceled by user`, `Failed assertion`, `Timed out` | |
126126

@@ -188,7 +188,7 @@ This event represents a finished testSuite execution. The event will contain the
188188
| id | `String` | Uniquely identifies the subject within the source. | `unit`, `e2e`, `security` | ✅ |
189189
| source | `URI-Reference` | [source](spec.md#source--context-) from the context | | |
190190
| environment | `Object` [`environment`](continuous-deployment.md/#environment) | The environment in which this testSuiteRun was running | `{"id": "1234"}`, `{"id": "dev", "source": "testkube-dev-123"}` | ✅ |
191-
| outcome | `String (enum)` | The outcome of the testSuite execution, one of `pass`, `fail`, `cancel`, `error` | `fail` | ✅ |
191+
| outcome | `String (enum)` | The outcome of the testSuite execution, one of `success`, `failure`, `cancel`, or `error` | `success`, `failure`, `cancel`, `error` | ✅ |
192192
| severity | `String (enum)` | Severity if the test failed, one of `low`, `medium`, `high`, `critical` | `critical`, `low`, `medium`, `high` |
193193
| reason | `String` | A reason related to the outcome of the execution | `Canceled by user`, `Failed testCase` | |
194194
| testSuite | `Object` [`testSuite`](#testsuite) | Definition of the testSuite being executed | `{"id": "92834723894", "name": "Auth TestSuite"}` | |

0 commit comments

Comments
 (0)