Skip to content

Commit b2b34fb

Browse files
Doc changes for restructured object (#334)
* Doc update for restructured object
1 parent 2483d1f commit b2b34fb

8 files changed

+65
-87
lines changed

wiki/markdown/confidence-level-modified-event-aggregation.md

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,7 @@ the database, but it was already aggregated with some other objects. The
7878
fetched object looks like that:
7979

8080
{
81-
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
82-
"aggregatedObject": {
81+
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
8382
"fileInformation": [
8483
{
8584
"extension": "jar",
@@ -211,8 +210,7 @@ fetched object looks like that:
211210
],
212211
"time": 1481875921763
213212
}
214-
]
215-
}
213+
]
216214
}
217215

218216
The required content is extracted from the event as specified in the rule:
@@ -239,8 +237,7 @@ format will look like below:
239237
And the resulting aggregated object will be:
240238

241239
{
242-
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
243-
"aggregatedObject": {
240+
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
244241
"fileInformation": [
245242
{
246243
"extension": "jar",
@@ -380,7 +377,6 @@ And the resulting aggregated object will be:
380377
"time": 1481875921763
381378
}
382379
]
383-
}
384380
}
385381

386382
### 2) f37d59a3-069e-4f4c-8cc5-a52e73501a75
@@ -458,8 +454,7 @@ aggregations were done during the time between this and previous
458454
ConfidenceLevelModified event appearance. Because of that the object looks like this:
459455

460456
{
461-
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
462-
"aggregatedObject": {
457+
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
463458
"fileInformation": [
464459
{
465460
"extension": "jar",
@@ -607,7 +602,6 @@ ConfidenceLevelModified event appearance. Because of that the object looks like
607602
"time": 1481875921763
608603
}
609604
]
610-
}
611605
}
612606

613607
The required content is extracted from the event as specified in the rule:
@@ -651,8 +645,7 @@ array. New “confidenceLevels” array will look like below:
651645
And the resulting aggregated object will be:
652646

653647
{
654-
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
655-
"aggregatedObject": {
648+
"_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
656649
"fileInformation": [
657650
{
658651
"extension": "jar",
@@ -806,5 +799,4 @@ And the resulting aggregated object will be:
806799
"time": 1481875921763
807800
}
808801
]
809-
}
810802
}

wiki/markdown/configuration.md

Lines changed: 0 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -63,21 +63,6 @@ Event repository. Recommended settings is 10 minutes.
6363

6464
* aggregated.collection.ttlValue (*seconds*)
6565

66-
When performing queries towards an aggregated object, **search.query.prefix**
67-
is used to access the aggregated object when using JMESPath expressions.
68-
69-
Consider the following query:
70-
71-
{
72-
"criteria": {
73-
"object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
74-
}
75-
}
76-
77-
Eiffel Intelligence will replace the prefix "object." with the name of the aggregated
78-
object (in this case "aggregatedObject"), which is defined by **aggregated.object.name**.
79-
Read more examples about [queries here](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/query.md).
80-
8166

8267
### Testing aggregation rules
8368

wiki/markdown/query.md

Lines changed: 23 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ Examples of this endpoint using curl
1717
curl -X GET -H "Content-type: application/json" http://localhost:8090/queryAggregatedObject?ID=6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43
1818

1919
## Perform freestyle query on created aggregated object
20-
It is possible to query for documents using freestyle queries. These freestyle
20+
It is possible to make queries on aggregated objects using freestyle queries. These freestyle
2121
queries are plain Mongo DB queries, you can read more about that [here](https://docs.mongodb.com/manual/tutorial/query-documents/).
2222

2323
POST /query
@@ -36,39 +36,39 @@ Examples of this endpoint using curl
3636

3737
Examples of criterias:
3838

39-
// This returns all objects where the object.testCaseExecutions.outcome.id is"TC5"
40-
// or object.testCaseExecutions.outcome.id is "TC6" and "object.identity"
39+
// This returns all objects where the testCaseExecutions.outcome.id is"TC5"
40+
// or testCaseExecutions.outcome.id is "TC6" and "identity"
4141
// is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
4242

4343
{
4444
"criteria": {
4545
"$or": [
4646
{
47-
"object.testCaseExecutions.outcome.id":"TC5"
47+
"testCaseExecutions.outcome.id":"TC5"
4848
}, {
49-
"object.testCaseExecutions.outcome.id":"TC6"
49+
"testCaseExecutions.outcome.id":"TC6"
5050
}
5151
],
52-
"object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
52+
"identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
5353
}
5454
}
5555

5656

5757
// This returns all objects where TC5 succeeded in
58-
// the array testCaseExecutions and object.identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
58+
// the array testCaseExecutions and identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
5959
// Options section is optional.
6060

6161
{
6262
"criteria": {
63-
"object.testCaseExecutions": {
63+
"testCaseExecutions": {
6464
"$elemMatch": {
6565
"outcome.conclusion": "SUCCESSFUL",
6666
"outcome.id": "TC5"
6767
}
6868
}
6969
},
7070
"options": {
71-
"object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
71+
"identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
7272
}
7373
}
7474

@@ -77,7 +77,7 @@ Examples of criterias:
7777
// and are related to JIRA ticket JIRA-1234
7878
{
7979
"criteria": {
80-
"object.testCaseExecutions": {
80+
"testCaseExecutions": {
8181
"$elemMatch": {
8282
"outcome.conclusion": "SUCCESSFUL",
8383
"outcome.id": "TC5"
@@ -105,10 +105,6 @@ Examples of criterias:
105105
}
106106
}
107107

108-
NOTE: It should be noted that "object" is a key word. It is replaced
109-
dynamically in the code, and is configured in the application.properties file with
110-
the search.query.prefix option. e.g., in string "object.testCaseExecutions.outcome.id",
111-
substring "object" is a key word.
112108

113109
## Example of freestyle query that returns all aggregated objects
114110
By using a query that contains only empty "criteria" it is possible to return
@@ -127,30 +123,30 @@ Example:
127123

128124

129125
## Query an aggregated object and filter it with specific key
130-
It is possible to filter the object and return only values with specific key or
131-
path. To do this, it is required to add filter condition to the json body. The
132-
parameter of filter condition is a JMESPath expression, you can read more about
133-
that [here](http://jmespath.org/tutorial.html#pipe-expressions).
126+
It is possible to filter the result from the query and return only the values
127+
which are of interest. This filter can be defined as a path in the aggregated
128+
object, or a specific key. The parameter of filter condition is a JMESPath
129+
expression, you can read more about that
130+
[here](http://jmespath.org/tutorial.html#pipe-expressions).
134131

135132
Example:
136133

137134
// This match an object in the same way as in the previous example. And then filter
138-
// it and returns eventId that has a path "aggregatedObject.publications[0].eventId".
139-
// aggregatedObject is the name of the aggregated object in the database.
135+
// it and returns eventId that has a path "publications[0].eventId".
140136

141137
{
142138
"criteria": {
143-
"object.testCaseExecutions": {
139+
"testCaseExecutions": {
144140
"$elemMatch": {
145141
"outcome.conclusion": "SUCCESSFUL",
146142
"outcome.id": "TC5"
147143
}
148144
}
149145
},
150146
"options": {
151-
"object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
147+
"identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
152148
},
153-
"filter" : "aggregatedObject.publications[0].eventId"
149+
"filter" : "publications[0].eventId"
154150
}
155151

156152

@@ -160,13 +156,13 @@ To filter with only the key or the partial path, it is required to use
160156
Example:
161157

162158
// This finds all objects where identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
163-
// Then it filters those objects and returns all values that has "svnIdentifier" as a key.
159+
// Then it filters those objects and returns all values that has "gitIdentifier" as a key.
164160

165161
{
166162
"criteria": {
167-
"object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
163+
"identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
168164
},
169-
"filter" : "incomplete_path_filter(@, 'svnIdentifier')"
165+
"filter" : "incomplete_path_filter(@, 'gitIdentifier')"
170166
}
171167

172168
As the filter functionality takes a plain jmespath expression it can be a more
@@ -183,7 +179,7 @@ Example
183179
"criteria": {
184180
"$where": "function(){ var searchKey = 'id'; var value = 'JIRA-1234'; return searchInObj(obj); function searchInObj(obj){ for (var k in obj){ if (obj[k] == value && k == searchKey) { return true; } if (isObject(obj[k]) && obj[k] !== null) { if (searchInObj(obj[k])) { return true;}}} return false; }}"
185181
},
186-
"filter": "{id: aggregatedObject.id, artifactIdentity:aggregatedObject.identity, confidenceLevels:aggregatedObject.confidenceLevels[?value=='SUCCESS'].{name: name, value: value}}"
182+
"filter": "{id: id, artifactIdentity: identity, confidenceLevels: confidenceLevels[?value=='SUCCESS'].{name: name, value: value}}"
187183
}
188184

189185
## Query missed notifications

wiki/markdown/rules.md

Lines changed: 27 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -70,22 +70,27 @@ types to discard, you will need 3 set of rules. Each rule might contain some of
7070
these keys, that have not null or empty values. Explicit list of rule keys are
7171
below:
7272

73-
**TemplateName** - used for specifying a template group, any string you like to
73+
### TemplateName
74+
used for specifying a template group, any string you like to
7475
name your template
7576

76-
**Type** - Eiffel event type, which will be used to find the matching rule set
77+
### Type
78+
Eiffel event type, which will be used to find the matching rule set
7779
for the received Eiffel event while creating aggregated object. Example:
7880
"EiffelConfidenceLevelModifiedEvent"
7981

80-
**TypeRule**- JMESPath identifier for the location of Eiffel event type in
82+
### TypeRule
83+
JMESPath identifier for the location of Eiffel event type in
8184
received Eiffel event. Example: "meta.type".
8285

83-
**IdRule** - JMESPath identifier for the location of Eiffel event id in
86+
### IdRule
87+
JMESPath identifier for the location of Eiffel event id in
8488
received Eiffel event. Example: "meta.id". Used as fall back when storing in
8589
database and no id is provided or to link id of aggregated objects to events
8690
that contributed to the aggregated object.
8791

88-
**StartEvent** - denotes if this event type starts the object aggregation. If
92+
### StartEvent
93+
denotes if this event type starts the object aggregation. If
8994
StartEvent is "YES" then it will be the first processed event in the aggregation
9095
sequence. If StartEvent is "NO" then Eiffel event of this type will be used to
9196
append information to existing aggregated object. If no aggregated object exist
@@ -95,7 +100,8 @@ then it will wait a certain time as defined by property
95100
created in time then the event will no longer be processed and it will be
96101
removed from the wait list.
97102

98-
**IdentifyRules** - JMESPath identifier of ids that will be used to search for
103+
### IdentifyRules
104+
JMESPath identifier of ids that will be used to search for
99105
an existing aggregated object. Should produce an JSON array, for example
100106
"[meta.id]"
101107

@@ -136,15 +142,17 @@ will return
136142
"cfce572b-c3j4-441e-abc9-b62f48080ca2"
137143
]
138144

139-
**MatchIdRules** - This rule is used to search in database containing the
145+
### MatchIdRules
146+
This rule is used to search in database containing the
140147
aggregated objects. The syntax of this rule is specific to querying in Mongo DB.
141148
Use marker "%IdentifyRules_objid%" to inject the ids generated by
142149
**_IdentifyRules_**. Examples:
143150

144151
{"_id": "%IdentifyRules_objid%"},
145-
{ "$and": [{"aggregatedObject.testCaseExecutions.testCaseStartedEventId": "%IdentifyRules%"}]}
152+
{ "$and": [{"testCaseExecutions.testCaseStartedEventId": "%IdentifyRules%"}]}
146153

147-
**ExtractionRules** - JSON object of JMESPath identifier(s) which will create
154+
### ExtractionRules
155+
JSON object of JMESPath identifier(s) which will create
148156
or modify the existing data in aggregated object. For a start event the content
149157
extracted with this rule will be the initial aggregated object.
150158

@@ -174,16 +182,23 @@ the JSON object
174182
which could be added to the root of aggregated object or to the inner structure
175183
of aggregated object depending on **_MergeResolverRules_**.
176184

177-
**MergeResolverRules** - This is a JSON object describing the entire path or
185+
### MergeResolverRules
186+
This is a JSON object describing the entire path or
178187
parts of the path to where the extracted content will be stored. More detailed
179188
explanation can be found [here](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/merge-resolver-rules.md)
180189

181-
**ProcessRules** - This rule is used to append (_only additions or modifications,
190+
### ProcessRules
191+
This rule is used to append (_only additions or modifications,
182192
no deletions, of existing content at time rule is applied_) new key value pairs
183193
in the aggregated object based on existing values in the aggregation object.
184194
For example if you have aggregated the job finished time and job started time
185195
then you can create one more value for the computed duration so that other
186196
systems do not need to compute it.
187197

188-
**History Rules** are used if you need to aggregate data from existing events
198+
### History Rules
199+
are used if you need to aggregate data from existing events
189200
linked upstream by received event. They are explained [**here**](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/history-rules.md)
201+
202+
## Reserved Key names
203+
The keys "_id" and "time" (at the root level of the aggregated object) are reserved and added in the mongo document by Eiffel Intelligence. User should not use these two keys at the
204+
root level of an aggregated object. Even If user add these two keys at the root level, Eiffel Intelligence will overwrite those.

wiki/markdown/subscriptions.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -57,12 +57,13 @@ subscription to trigger. Subscription templates [can be found here](https://gith
5757
We will go through how to write some example requirements in subscriptions
5858
based on the below aggregated object. Conditions in subscriptions are
5959
written with [JMESPath syntax](http://jmespath.org/specification.html).
60-
Below is a document from the database which contains a unique ID and an
61-
aggregated object.
60+
Below is a document containing an aggregated object. In addition to the
61+
extracted content from Eiffel events, Eiffel Intelligence has added an
62+
extra _id key with the value of the start event. This _id serves as an
63+
index in mongo DB and speeds up searches.
6264

6365
{
64-
"_id" : "df4cdb42-1580-4cff-b97a-4d0faa9b2b22",
65-
"aggregatedObject" : {
66+
"_id" : "df4cdb42-1580-4cff-b97a-4d0faa9b2b22",
6667
"fileInformation" : [
6768
{
6869
"extension" : "war",
@@ -96,7 +97,6 @@ aggregated object.
9697
"time" : 1521452368758
9798
}
9899
]
99-
}
100100
}
101101

102102
Let's say we need a subscription which is triggered on when an artifact

wiki/markdown/test-case-finished-event-aggregation.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -145,8 +145,7 @@ Finally, you will be able to find the fully aggregated object that may contain
145145
information about one or several executed test cases:
146146

147147
{
148-
"_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
149-
"aggregatedObject":{
148+
"_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
150149
"fileInformation":[
151150
{
152151
"extension":"jar",
@@ -305,5 +304,4 @@ information about one or several executed test cases:
305304
"identity": "pkg:maven/com.othercompany.secondproduct/other-system@1.33.0"
306305
}
307306
]
308-
}
309307
}

wiki/markdown/test-case-started-event-aggregation.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,8 +64,7 @@ Following aggregated object is extracted by using the identify rules:
6464

6565
[
6666
{
67-
"_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
68-
"aggregatedObject":{
67+
"_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
6968
"fileInformation":[
7069
{
7170
"extension":"jar",
@@ -106,7 +105,6 @@ Following aggregated object is extracted by using the identify rules:
106105
"time":1481875891763,
107106
"type":"EiffelArtifactCreatedEvent",
108107
"identity": "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
109-
}
110108
}
111109
]
112110

0 commit comments

Comments
 (0)