Doc changes for restructured object (#334)

* Doc update for restructured object

Author: saif-ericsson

wiki/markdown/confidence-level-modified-event-aggregation.md

@@ -78,8 +78,7 @@ the database, but it was already aggregated with some other objects. The
 fetched object looks like that:
 
     {
-        "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-        "aggregatedObject": {
+          "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
           "fileInformation": [
             {
               "extension": "jar",
@@ -211,8 +210,7 @@ fetched object looks like that:
               ],
               "time": 1481875921763
             }
-          ]
-        }
+          ]        
       }
 
 The required content is extracted from the event as specified in the rule:
@@ -239,8 +237,7 @@ format will look like below:
 And the resulting aggregated object will be:
 
     {
-      "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-      "aggregatedObject": {
+        "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
         "fileInformation": [
           {
             "extension": "jar",
@@ -380,7 +377,6 @@ And the resulting aggregated object will be:
             "time": 1481875921763
           }
         ]
-      }
     }
 
 ### 2) f37d59a3-069e-4f4c-8cc5-a52e73501a75
@@ -458,8 +454,7 @@ aggregations were done during the time between this and previous
 ConfidenceLevelModified event appearance. Because of that the object looks like this:
 
     {
-        "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-        "aggregatedObject": {
+          "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
           "fileInformation": [
             {
               "extension": "jar",
@@ -607,7 +602,6 @@ ConfidenceLevelModified event appearance. Because of that the object looks like
               "time": 1481875921763
             }
           ]
-        }
       }
 
 The required content is extracted from the event as specified in the rule:
@@ -651,8 +645,7 @@ array. New “confidenceLevels” array will look like below:
 And the resulting aggregated object will be:
 
     {
-      "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-      "aggregatedObject": {
+        "_id": "6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
         "fileInformation": [
           {
             "extension": "jar",
@@ -806,5 +799,4 @@ And the resulting aggregated object will be:
             "time": 1481875921763
           }
         ]
-      }
     }

wiki/markdown/configuration.md

@@ -63,21 +63,6 @@ Event repository. Recommended settings is 10 minutes.
 
 * aggregated.collection.ttlValue (*seconds*)
 
-When performing queries towards an aggregated object, **search.query.prefix**
-is used to access the aggregated object when using JMESPath expressions.
-
-Consider the following query:
-
-    {
-      "criteria": {
-        "object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
-      }
-    }
-
-Eiffel Intelligence will replace the prefix "object." with the name of the aggregated
-object (in this case "aggregatedObject"), which is defined by **aggregated.object.name**.
-Read more examples about [queries here](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/query.md).
-
 
 ### Testing aggregation rules
 

wiki/markdown/query.md

@@ -17,7 +17,7 @@ Examples of this endpoint using curl
     curl -X GET -H "Content-type: application/json"  http://localhost:8090/queryAggregatedObject?ID=6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43
 
 ## Perform freestyle query on created aggregated object
-It is possible to query for documents using freestyle queries. These freestyle
+It is possible to make queries on aggregated objects using freestyle queries. These freestyle
 queries are plain Mongo DB queries, you can read more about that [here](https://docs.mongodb.com/manual/tutorial/query-documents/).
 
     POST /query
@@ -36,39 +36,39 @@ Examples of this endpoint using curl
 
 Examples of criterias:
 
-    // This returns all objects where the object.testCaseExecutions.outcome.id is"TC5"
-    // or object.testCaseExecutions.outcome.id is "TC6" and "object.identity"
+    // This returns all objects where the testCaseExecutions.outcome.id is"TC5"
+    // or testCaseExecutions.outcome.id is "TC6" and "identity"
     // is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
 
     {
       "criteria": {
         "$or": [
           {
-            "object.testCaseExecutions.outcome.id":"TC5"
+            "testCaseExecutions.outcome.id":"TC5"
           }, {
-            "object.testCaseExecutions.outcome.id":"TC6"
+            "testCaseExecutions.outcome.id":"TC6"
           }
         ],
-        "object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
+        "identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
       }
     }
 
 
     // This returns all objects where TC5 succeeded in
-    // the array testCaseExecutions and object.identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
+    // the array testCaseExecutions and identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
     // Options section is optional.
 
     {
       "criteria": {
-        "object.testCaseExecutions": {
+        "testCaseExecutions": {
           "$elemMatch": {
             "outcome.conclusion": "SUCCESSFUL",
             "outcome.id": "TC5"
           }
         }
       },
       "options": {
-        "object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
+        "identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
       }
     }
 
@@ -77,7 +77,7 @@ Examples of criterias:
     // and are related to JIRA ticket JIRA-1234
     {
         "criteria": {
-             "object.testCaseExecutions": {
+             "testCaseExecutions": {
                  "$elemMatch": {
                      "outcome.conclusion": "SUCCESSFUL",
                      "outcome.id": "TC5"
@@ -105,10 +105,6 @@ Examples of criterias:
         }
     }
 
-NOTE: It should be noted that "object" is a key word.  It is replaced
-dynamically in the code, and is configured in the application.properties file with
-the search.query.prefix option. e.g., in string "object.testCaseExecutions.outcome.id",
-substring "object" is a key word.
 
 ## Example of freestyle query that returns all aggregated objects
 By using a query that contains only empty "criteria" it is possible to return
@@ -127,30 +123,30 @@ Example:
 
 
 ## Query an aggregated object and filter it with specific key
-It is possible to filter the object and return only values with specific key or
-path. To do this, it is required to add filter condition to the json body. The
-parameter of filter condition is a JMESPath expression, you can read more about
-that [here](http://jmespath.org/tutorial.html#pipe-expressions).
+It is possible to filter the result from the query and return only the values
+which are of interest. This filter can be defined as a path in the aggregated
+object, or a specific key. The parameter of filter condition is a JMESPath 
+expression, you can read more about that
+[here](http://jmespath.org/tutorial.html#pipe-expressions).
 
 Example:
 
     // This match an object in the same way as in the previous example. And then filter
-    // it and returns eventId that has a path "aggregatedObject.publications[0].eventId".
-    // aggregatedObject is the name of the aggregated object in the database.
+    // it and returns eventId that has a path "publications[0].eventId".
 
     {
       "criteria": {
-        "object.testCaseExecutions": {
+        "testCaseExecutions": {
           "$elemMatch": {
             "outcome.conclusion": "SUCCESSFUL",
             "outcome.id": "TC5"
           }
         }
       },
       "options": {
-        "object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
+        "identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
       },
-      "filter" : "aggregatedObject.publications[0].eventId"
+      "filter" : "publications[0].eventId"
     }
 
 
@@ -160,13 +156,13 @@ To filter with only the key or the partial path, it is required to use
 Example:
 
     // This finds all objects where identity is "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0".
-    // Then it filters those objects and returns all values that has "svnIdentifier" as a key.
+    // Then it filters those objects and returns all values that has "gitIdentifier" as a key.
 
     {
       "criteria": {
-         "object.identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
+         "identity":"pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
       },
-      "filter" : "incomplete_path_filter(@, 'svnIdentifier')"
+      "filter" : "incomplete_path_filter(@, 'gitIdentifier')"
     }
 
 As the filter functionality takes a plain jmespath expression it can be a more
@@ -183,7 +179,7 @@ Example
         "criteria": {
             "$where": "function(){ var searchKey = 'id'; var value = 'JIRA-1234'; return searchInObj(obj); function searchInObj(obj){ for (var k in obj){ if (obj[k] == value && k == searchKey) { return true;  } if (isObject(obj[k]) && obj[k] !== null) { if (searchInObj(obj[k])) { return true;}}} return false; }}"
         },
-        "filter": "{id: aggregatedObject.id, artifactIdentity:aggregatedObject.identity, confidenceLevels:aggregatedObject.confidenceLevels[?value=='SUCCESS'].{name: name, value: value}}"
+        "filter": "{id: id, artifactIdentity: identity, confidenceLevels: confidenceLevels[?value=='SUCCESS'].{name: name, value: value}}"
     }
 
 ## Query missed notifications

wiki/markdown/rules.md

@@ -70,22 +70,27 @@ types to discard, you will need 3 set of rules. Each rule might contain some of
 these keys, that have not null or empty values. Explicit list of rule keys are
 below:
 
-**TemplateName** - used for specifying a template group, any string you like to
+### TemplateName 
+used for specifying a template group, any string you like to
 name your template
 
-**Type** - Eiffel event type, which will be used to find the matching rule set
+### Type 
+Eiffel event type, which will be used to find the matching rule set
 for the received Eiffel event while creating aggregated object. Example:
 "EiffelConfidenceLevelModifiedEvent"
 
-**TypeRule**- JMESPath identifier for the location of Eiffel event type in
+### TypeRule 
+JMESPath identifier for the location of Eiffel event type in
 received Eiffel event. Example: "meta.type".
 
-**IdRule** - JMESPath identifier for the location of Eiffel event id in
+### IdRule 
+JMESPath identifier for the location of Eiffel event id in
 received Eiffel event. Example: "meta.id". Used as fall back when storing in
 database and no id is provided or to link id of aggregated objects to events
 that contributed to the aggregated object.
 
-**StartEvent** - denotes if this event type starts the object aggregation. If
+### StartEvent
+denotes if this event type starts the object aggregation. If
 StartEvent is "YES" then it will be the first processed event in the aggregation
 sequence. If StartEvent is "NO" then Eiffel event of this type will be used to
 append information to existing aggregated object. If no aggregated object exist
@@ -95,7 +100,8 @@ then it will wait a certain time as defined by property
  created in time then the event will no longer be processed and it will be
  removed from the wait list.
 
-**IdentifyRules** - JMESPath identifier of ids that will be used to search for
+### IdentifyRules
+JMESPath identifier of ids that will be used to search for
 an existing aggregated object. Should produce an JSON array, for example
     "[meta.id]"
 
@@ -136,15 +142,17 @@ will return
      "cfce572b-c3j4-441e-abc9-b62f48080ca2"
     ]
 
-**MatchIdRules** - This rule is used to search in database containing the
+### MatchIdRules 
+This rule is used to search in database containing the
 aggregated objects. The syntax of this rule is specific to querying in Mongo DB.
 Use marker "%IdentifyRules_objid%" to inject the ids generated by
 **_IdentifyRules_**. Examples:
 
     {"_id": "%IdentifyRules_objid%"},
-    { "$and": [{"aggregatedObject.testCaseExecutions.testCaseStartedEventId": "%IdentifyRules%"}]}
+    { "$and": [{"testCaseExecutions.testCaseStartedEventId": "%IdentifyRules%"}]}
 
-**ExtractionRules** - JSON object of JMESPath identifier(s) which will create
+### ExtractionRules
+JSON object of JMESPath identifier(s) which will create
 or modify the existing data in aggregated object. For a start event the content
 extracted with this rule will be the initial aggregated object.
 
@@ -174,16 +182,23 @@ the JSON object
 which could be added to the root of aggregated object or to the inner structure
 of aggregated object depending on **_MergeResolverRules_**.
 
-**MergeResolverRules** - This is a JSON object describing the entire path or
+### MergeResolverRules
+This is a JSON object describing the entire path or
 parts of the path to where the extracted content will be stored. More detailed
 explanation can be found [here](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/merge-resolver-rules.md)
 
-**ProcessRules** - This rule is used to append (_only additions or modifications,
+### ProcessRules 
+This rule is used to append (_only additions or modifications,
 no deletions, of existing content at time rule is applied_) new key value pairs
 in the aggregated object based on existing values in the aggregation object.
 For example if you have aggregated the job finished time and job started time
 then you can create one more value for the computed duration so that other
 systems do not need to compute it.
 
-**History Rules** are used if you need to aggregate data from existing events
+### History Rules 
+are used if you need to aggregate data from existing events
 linked upstream by received event. They are explained [**here**](https://github.com/eiffel-community/eiffel-intelligence/blob/master/wiki/markdown/history-rules.md)
+
+## Reserved Key names
+The keys "_id" and "time" (at the root level of the aggregated object) are reserved and added in the mongo document by Eiffel Intelligence. User should not use these two keys at the 
+root level of an aggregated object. Even If user add these two keys at the root level, Eiffel Intelligence will overwrite those.

wiki/markdown/subscriptions.md

@@ -57,12 +57,13 @@ subscription to trigger. Subscription templates [can be found here](https://gith
 We will go through how to write some example requirements in subscriptions
 based on the below aggregated object. Conditions in subscriptions are
 written with [JMESPath syntax](http://jmespath.org/specification.html).
-Below is a document from the database which contains a unique ID and an
-aggregated object.
+Below is a document containing an aggregated object. In addition to the 
+extracted content from Eiffel events, Eiffel Intelligence has added an 
+extra _id key with the value of the start event. This _id serves as an 
+index in mongo DB and speeds up searches.
 
     {
-        "_id" : "df4cdb42-1580-4cff-b97a-4d0faa9b2b22",
-        "aggregatedObject" : {
+            "_id" : "df4cdb42-1580-4cff-b97a-4d0faa9b2b22",
             "fileInformation" : [
                 {
                     "extension" : "war",
@@ -96,7 +97,6 @@ aggregated object.
                     "time" : 1521452368758
                 }
             ]
-        }
     }
 
 Let's say we need a subscription which is triggered on when an artifact

wiki/markdown/test-case-finished-event-aggregation.md

@@ -145,8 +145,7 @@ Finally, you will be able to find the fully aggregated object that may contain
 information about one or several executed test cases:
 
     {
-      "_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-      "aggregatedObject":{
+        "_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
         "fileInformation":[
           {
             "extension":"jar",
@@ -305,5 +304,4 @@ information about one or several executed test cases:
             "identity": "pkg:maven/com.othercompany.secondproduct/other-system@1.33.0"
           }
         ]
-      }
     }

wiki/markdown/test-case-started-event-aggregation.md

@@ -64,8 +64,7 @@ Following aggregated object is extracted by using the identify rules:
 
     [
         {
-           "_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
-           "aggregatedObject":{
+              "_id":"6acc3c87-75e0-4b6d-88f5-b1a5d4e62b43",
               "fileInformation":[
                  {
                     "extension":"jar",
@@ -106,7 +105,6 @@ Following aggregated object is extracted by using the identify rules:
               "time":1481875891763,
               "type":"EiffelArtifactCreatedEvent",
               "identity": "pkg:maven/com.mycompany.myproduct/sub-system@1.1.0"
-           }
         }
     ]