Merge pull request #6 from HP4k1h5/v0.1.1

V0.1.1

Author: hp4k1h5

CHANGELOG.md

@@ -0,0 +1,29 @@
+# CHANGELOG
+
+- v0.1.1
+  - ❌ Breaking change!  
+      `buildAQL()`'s `limit` parameter no longer accepts key
+      `end`, which has been renamed, per Arango spec, to `count`. The functionality
+      remains the same, which is why the patch bump. Please accept my apologies
+      for the un-semver approach. I have a personal philosophy that v0.X.X is
+      not really subject to the standard rules of semver, but I will do my best
+      to not present further breaking changes without upping at least the minor
+      version. It's important for any user of the library to have a version that
+      will be more compatible with all future versions going forward.
+  - 🔑🗝 multi-key search  
+      this can be useful if you
+      have multiple fields with textual information. Theoretically, each
+      chapter of a book could be stored on its own key. Or a document could
+      have be translated into several languages, each stored on its own key.
+      - **There are two ways to provide multiple keys to relevant functions.**
+      1) `query.key` now accepts in addition to a string value, an array of
+      strings over which the query is to be run. All keys listed here will be
+      run combined with all collections provided to `query.collections`
+      **unless** a collection has a `keys` property of its own, in which case
+      **only** those keys are searched against.
+      2) `query.collections[].keys` is an optional array of key names that are
+      indexed by `query.collections[].analyzer`. **Note** It is important that
+      any key listed in `query.collections[].keys` be indexed by the analyzer
+      as it will impact results if such a key does not exist.  
+
+

README.md

@@ -1,24 +1,51 @@
 # AQLqueryBuilder.js
 > a typescript query builder for [arangodb](https://www.arangodb.com)'s [ArangoSearch](https://www.arangodb.com/docs/stable/arangosearch.html)
-See working demo at [hp4k1h5.github.io](https://hp4k1h5.github.io/#search-box).  
+
+See working demo at [hp4k1h5.github.io](https://hp4k1h5.github.io/#search-box).
 
 **!Note** AQLqueryBuilder.js does NOT contain any code for the search bar, only the
 query string parser and AQL builder. Unabstracted code for the searchbar is
-located
-[here](https://github.com/HP4k1h5/hp4k1h5.github.io/tree/main/demos/src/components/search).
+located [here](https://github.com/HP4k1h5/hp4k1h5.github.io/tree/main/demos/src/components/search).
+
 ![search bar demonstration with schematic query
 interface](./img/searchbar_demo.png)
 
-- [ overview ](#overview)
-- [ setup](#setup)
-  - [ installation](#installation)
-- [ usage](#usage)
-  - [ query object](#query-object)
-  - [ boolean search logic](#boolean-search-logic)
-  - [ default query syntax](#default-query-syntax)
-  - [ Example](#Example)
-- [ bugs](#bugs)
-- [ contributing](#contributing)
+- [Patch Notes](#patch-notes)
+- [overview](#overview)
+- [setup](#setup)
+- [installation](#installation)
+- [usage](#usage)
+  - [`buildAQL()`](#buildaql())
+  - [boolean search logic](#boolean-search-logic)
+  - [default query syntax](#default-query-syntax)
+    - [Example](#example)
+- [bugs](#bugs)
+- [contributing](#contributing)
+
+## Patch Notes
+
+- v0.1.1
+  - ❌ Breaking change!  
+      `buildAQL()`'s `limit` parameter no longer accepts key
+      `end`, which has been renamed, per Arango spec, to `count`. The functionality
+      remains the same, which is why the patch bump. Please accept my apologies.
+  - 🔑🗝 multi-key search  
+      this can be useful if you
+      have multiple fields with textual information. Theoretically, each
+      chapter of a book could be stored on its own key. Or a document could
+      have be translated into several languages, each stored on its own key.
+      - **There are two ways to provide multiple keys to relevant functions.**
+      1) `query.key` now accepts in addition to a string value, an array of
+      strings over which the query is to be run. All keys listed here will be
+      run combined with all collections provided to `query.collections`
+      **unless** a collection has a `keys` property of its own, in which case
+      **only** those keys are searched against.
+      2) `query.collections[].keys` is an optional array of key names that are
+      indexed by `query.collections[].analyzer`. **Note** It is important that
+      any key listed in `query.collections[].keys` be indexed by the analyzer
+      as it will impact results if such a key does not exist.  
+
+
 
 ## overview
 
@@ -87,7 +114,8 @@ also accommodate multiple key searches.
 ___
 ## setup
 
-1) running generated AQL queries will require a running arangodb instance.
+1) running generated AQL queries will require a running ArangoDB v3.6+
+instance. This package is tested against v3.6.5 🥑
 
 ## installation
 currently there is only support for server-side use.
@@ -96,7 +124,8 @@ currently there is only support for server-side use.
     or `npm install --save @hp4k1h5/AQLqueryBuilder.js`  
     in a directory containing a `package.json` file.  
    __or__  
-  clone this repository in your node compatible project.
+  clone this repository in your node compatible project. And run `yarn` from
+  inside the directory.
 
 2) import/require the exported functions
 ```js
@@ -113,22 +142,15 @@ const {buildAQL} = require('@hp4k1h5/AQLqueryBuilder.js')
 __for up-to-date documentation, run `yarn doc && serve docs/` from the project
   directory root.__
 
-AQLqueryBuilder aims to provide cross-collection and cross-language boolean
-search capabilities to the library's user. Currently, this library makes a
-number of assumptions about the way your data is stored and indexed, but these
-are hopefully compatible with a wide range of setups.
-
-The primary assumption this library makes is that the data you are trying to
-query against is indexed by an ArangoSearch View, and that all documents index
-the same exact field. This field, passed to the builder as a key on the
-`query` object passed to e.g. `buildAQL()`, can be indexed by any number of
-analyzers, and the query will target all supplied collections simultaneously.
-This allows for true multi-language search, provided all analyzers and
-indexers index the same key as all other documents in the view. While there
-are plans to expand on this functionality to provide multi-key search, this
-library is primarily built for academic and textual searches, and is ideally
-suited for documents like books, articles, and other media where most of the
-data resides in a single place, i.e. document `key`, or `field`.
+AQLqueryBuilder aims to provide cross-collection and cross-language, multi-key
+boolean search capabilities to the library's user.
+
+Please ensure that the data you are trying to query against is indexed by an
+ArangoSearch View. The query will target all combinations of provided
+collections, analyzers and keys an simultaneously. This allows for granular
+multi-language search. This library is primarily built for academic and
+textual searches, and is ideally suited for documents like books, articles,
+and other text-heavy media.
 
 AQLqueryBuilder works best as a document query tool. Leveraging ArangoSearch's
 built-in language stemming analyzers allows for complex search phrases to be
@@ -143,55 +165,112 @@ const queryObject = {
   "view": "the_arango-search_view-name",
   "collections": [{
     "name": "collection_name",
-    "analyzer": "analyzer_name"
+    "analyzer": "analyzer_name",
+    "keys": ["text"]
   }],
   "query": "+'query string' -for +parseQuery ?to parse"
 }
 
-const aqlQuery = buildAQL(queryObject)
+const limit = {start:0, count: 20}
+
+const aqlQuery = buildAQL(queryObject, limit)
 
 // ... const cursor = await db.query(aqlQuery)
-// ... const cursor = await db.query(buildAQL(queryObject, {start:21, end:40})
+// ... const cursor = await db.query(buildAQL(queryObject, {start:20, count:20})
 ```
 
 Generate documenation with `yarn doc && serve docs/` or see more examples in
 e.g. [tests/search.ts](tests/search.ts)
 
-### query object
+___
+
+### `buildAQL()`
 
-`buildAQL` accepts an object with the following properties:
+`buildAQL` accepts two parameters: `query` and `limit`
 
+#### query
 • **view**: *string* (required): the name of the ArangoSearch view the query
 will be run against
 
-• **collections** (required): the names of the collections indexed by @view to query
-
-• **terms** (required): either an array of `term` interfaces or a string to be
-parsed by `parseQuery`
-
-• **key** (optional | default: "text"): the name of the Arango document key to search
-within.
+• **collections** (required): an array of objects of the indexed collections
+to query. Objects have the following shape:
+```json
+{
+  "name": "collection-name",
+  "analyzer": "analyzer_name",
+  "keys": ["text", "summary", "notes"]
+}
+```
+`keys` are optional, though if key names are provided to `query.key`, and
+not all those keys are indexed by the collection, it is advisable to
+explicitly list only those keys on documents in that collection that are
+indexed by the analyzer. If a collection is indexed by multiple analyzers
+please list each collection-analyzer combination along with their relevant
+keys, unless a unified set of keys is provided to `query.key`.
+
+• **query** (required): either an array of `term` interfaces or a query string
+to be parsed by `parseQuery`.
+- **term** (optional): a JSON object representing the search.
+  - **val** *string* (required): a search term or phrase. For PHRASE
+  searches, include one "quoted phrase" per val. For TOKENS you may combine
+  multiple terms under a single analyzer or use one `term` per token.
+  - **op** *string* (required): boolean operator; **one of `+ ? -`,**
+  representing `AND OR NOT` respectively
+  - **type** *string* (required): **one of `"phr"`** for PHRASES **or `"tok"`**
+  for TOKENS.  
+
+• **key** (optional | default: "text"): the name of the Arango document key to
+search within.
+
+• **filters** (optional): a list of filter interfaces. See [arango FILTER
+operations](https://www.arangodb.com/docs/stable/aql/operations-filter.html)
+for more details. All [Arango
+operators](https://www.arangodb.com/docs/3.6/aql/operators.html ) Filters have
+the following shape:
+```json
+{
+  "field": "the name of the field, i.e. Arango document key to filter on",
+  "op": "one of: == != < > <= >= IN NOT IN LIKE NOT LIKE =~ !~     ",
+  "val": "the "
+}
+```
 
-• **filters** (optional): a list of @filter interfaces
+#### limit
+an object with the following keys:
+- `start` (integer) 0-based result pagination lower bound.
+- `count` (integer) total number of results to return. ⚠ see CHANGELOG v0.1.1  
 
-___
+to bring back up to the first 10 results
+```json
+{"start":0, "count":10}
+```
+and the next page would be
+```json
+{"start":10, "count":10}
+```
 
-#### `query` Example
+#### `query` example
 ```json
 {
   "view": "the_arango-search_view-name",
   "collections": [
     {
-      "name":
-      "collection_name", "analyzer":
-       "analyzer_name"
+      "name":"collection_name",
+      "analyzer": "analyzer_name",
+      "keys": ["text", "summary", "notes"]
+    },
+    {
+      "name":"collection_es",
+      "analyzer": "analyzer_es",
+      "keys": ["texto", "resumen", "notas"]
     }
   ],
-  "key": "text",
-  "query": "either a +query ?\"string for parseQuery to parse\"",
+  "query": "either a +query ?\"string for parseQuery to parse\"    texto en español",
   "query": [
            {"type": "phr", "op": "?", "val": "\"!!! OR a list of query objects\""},
-           {"type": "tok", "op": "-", "val": "tokens"}
+           {"type": "phr", "op": "?", "val": "optional phrases"},
+           {"type": "tok", "op": "-", "val": "excluded tokens"},
+           {"type": "tok", "op": "+", "val": "mandatory tokens"}
          ],
   "filters": [
     {
@@ -203,12 +282,9 @@ ___
 }
 ```
 
-
 ___
 ### boolean search logic
-
 Quoting [mit's Database Search Tips](https://libguides.mit.edu/c.php?g=175963&p=1158594):
-
 > Boolean operators form the basis of mathematical sets and database logic.
     They connect your search words together to either narrow or broaden your
     set of results.  The three basic boolean operators are: AND, OR, and NOT.
@@ -289,5 +365,4 @@ ___
 please see [bugs](https://github.com/HP4k1h5/AQLqueryBuilder.js/issues/new?assignees=HP4k1h5&labels=bug&template=bug_report.md&title=basic)
 
 ## contributing
-please see [./.github/CONTRIBUTING.md]
-
+please see [CONTRIBUTING](./.github/CONTRIBUTING.md)

package.json

@@ -32,7 +32,7 @@
   },
   "devDependencies": {
     "@types/chai": "^4.2.11",
-    "@types/mocha": "^7.0.2",
+    "@types/mocha": "^8.0.0",
     "@types/node": "^14.0.13",
     "chai": "^4.2.0",
     "mocha": "^8.0.1",

src/index.ts

@@ -5,15 +5,20 @@ import { buildFilters } from './filter'
 export { buildFilters } from './filter'
 export { parseQuery } from './parse'
 
-/** @returns an AQL query object. See @param query for
- * details on required values. @param query.terms accepts
- * either a string to be parsed or an array of terms. @param limit is an object with keys `start` default 0, and `end` default 20.
- * */
+/** @returns an AQL query object. See @param query for details on required
+ * values. @param query.terms accepts either a string to be parsed or an array
+ * of terms.
+ *
+ * ! NOTE: v0.1.1 introduced a breaking change to adhere to the ArangoSearch
+ * spec. Please refer to
+ * <https://www.arangodb.com/docs/stable/aql/operations-limit.html> @param
+ * limit is an object with keys `start` @default 0, and `count` @default 20 */
 export function buildAQL(
   query: query,
-  limit: any = { start: 0, end: 20 },
+  limit: any = { start: 0, count: 20 },
 ): any {
   validateQuery(query)
+  collectKeys(query)
 
   const SEARCH = buildSearch(query)
   const FILTER = query.filters && buildFilters(query.filters)
@@ -23,7 +28,7 @@ export function buildAQL(
     FOR doc IN ${aql.literal(query.view)}
       ${SEARCH}
       ${FILTER}
-      LIMIT ${limit.start}, ${limit.end}
+      LIMIT ${limit.start}, ${limit.count}
     RETURN doc`
 }
 
@@ -33,3 +38,18 @@ function validateQuery(query: query) {
   if (!query.collections.length)
     throw new Error('query.collections must have at least one name')
 }
+
+function collectKeys(query: query) {
+  /* unify query.key */
+  let _keys: string[]
+  if (typeof query.key == 'string') {
+    _keys = [query.key]
+  } else if (!query.key) {
+    _keys = ['text']
+  } else _keys = query.key
+
+  query.collections = query.collections.map((c) => {
+    if (!c.keys) c.keys = _keys
+    return c
+  })
+}