docs: Improve docs content

Author: hyperupcall

docs/bobject-get.md

@@ -1,27 +1,21 @@
 # Errors
 
-Because Bash often does the wrong thing silently, it's important to fail early when any inputs are unexpected. Extra checks that require spawning subshells require setting the `VERIFY_BASH_OBJECT` variable. We recommend exporting it so any child Bash processes keep the verification behavior
+Because Bash often does the wrong thing silently, it's important to fail early. This page documents the types of errors `bash-object` catches when using its functionality
 
-The following checks are performed
+Export the `VERIFY_BASH_OBJECT` variable if you want to perform extra validation checks on input. On newer Bash versions this will incur little overhead; on older ones this will spawn subshells (which is why it's not by default enabled)
 
-1. Argument parsing
+## Error Categories
 
-- Ensures the correct number of arguments are passed (when applicable)
-- Ensures the arguments are not empty (when applicable)
-- Ensures applicable arguments refers to real variables (ex. the root object in both set/get, and the final_value for set)
-- Ensure applicable arguments refer to variables of the same type (for set)
+Generally, there are three categories of errors
 
-2. Query parsing
+### 1. Arguments
 
-- Ensures the query has the correct form
+Prefixed with `ERROR_ARGUMENTS_`, these error codes will show when there is something wrong with the passed flags or arguments. We will check to ensure the correct number of arguments have been passed (and are non empty). Additionally, we will ensure that the passed arguments have continuity with whatever you reference. For example, if you want to `--ref set-object`, we will ensure that the variable you specify is a defined variable, and actually is an object (associative array). Another example, if you wish to `set` or `get` a variable, it will check to ensure the variable at the place you specify with the querystring is the same type as you specified as an argument (e.g. `set-array`)
 
-3. Tree traversing
+### 2. Querytree
 
-- Ensure type specified in the virtual object matches that of the referenced object
-- Ensure type implied in query string matches the one specified in the virtual object
+Prefixed with `ERROR_QUERYTREE_`, these error codes will show when there is something wrong with parsing the querytree, namely syntax errors. A message associated with the error code will specify why the querytree couldn't be fixed, along with possible solutions
 
-4. Final operation (set/get)
+### 3. Virtual objects
 
-- Ensure type specified in the virtual object matches that of the referenced object
-- Ensure type implied in query string matches the one specified in the virtual object
-- Ensure type specified in set/get matches the one specified in the virtual object
+Prefixed with `ERROR_VOBJ_`, these error codes will show when the virtual object does not match up with its reference. For example, if a virtual object looks like `$'\x1C\x1Dtype=array;&SUB_OBJECT'`, an error will show because it is referencing an object with the variable name `SUB_OBJECT`, but `type=array` is specified

docs/bobject.set.md

@@ -0,0 +1,98 @@
+# Unmarshaling
+
+Unmarshaling strings representing highly structured data in Bash in a reliable and easy to use way has historically been impossible - until now
+
+The following examples will use JSON format, which represents the data that Bash will encode in-memory
+
+## Motivation
+
+Bash allows for simple/flat mapping via an associative array
+
+```json
+{
+	"atom": "Hydrogen"
+}
+```
+
+```bash
+assert [ "${OBJECT[atom]}" = 'Hydrogen' ]
+```
+
+But, when it comes to even the slightest of more complicated structures, Bash cannot cope: associative arrays do not nest inside one another
+
+```json
+{
+	"stars": {
+		"cool": "Wolf 359"
+	}
+}
+```
+
+Note only that, but indexed arrays cannot nest inside associative arrays
+
+```json
+{
+	"nearby": [
+		"Alpha Centauri A",
+		"Alpha Centauri B",
+		"Proxima Centauri",
+		"Barnard's Star",
+		"Luhman 16"
+	]
+}
+```
+
+## Solution
+
+`bash-object` solves this problem, in the most general case. The repository contains functions for storing and retreiving strings, indexed arrays, and associative arrays in a heterogenous hierarchy
+
+Let's take a look at the most basic case
+
+```json
+{
+	"xray": {
+		"yankee": "zulu"
+	}
+}
+```
+
+In Bash, it will be stored in memory eventually using declarations similar to the following
+
+```sh
+declare -A unique_global_variable_xray=([yankee]='zulu')
+declare -A OBJECT=([xray]=$'\x1C\x1Dtype=object;&unique_global_variable_xray')
+```
+
+You can retrieve the data with
+
+```sh
+bobject get-object --value 'OBJECT' 'xray'
+assert [ "${REPLY[yankee]}" = zulu ]
+```
+
+The implementation hinges on Bash's `declare -n`. When using `get-object`, this is what would happen behind the scenes at the lowest level
+
+```sh
+local current_object_name='unique_global_variable_xray'
+local -n current_object="$current_object_name"
+
+declare -gA REPLY=()
+local key=
+for key in "${!current_object[@]}"; do
+	REPLY["$key"]="${current_object["$key"]}"
+done
+```
+
+Another implementation detail is how a new variable is created in the global scope. This can occur when you are setting an array (indexed array) or object (associative array) at some place in the object hierarchy. In the previous example, `unique_global_variable_xray` is the new variable created in the global scope; in practice, the name looks a lot different, as seen in the example below
+
+```sh
+local global_object_name=
+printf -v global_object_name '%q' "__bash_object_${root_object_name}_${root_object_query}_${RANDOM}_${RANDOM}"
+
+if ! eval "declare -gA $global_object_name=()"; then
+	bash_object.util.die 'ERROR_INTERNAL' 'Eval declare failed'
+	return
+fi
+```
+
+Unfortunately, it must use eval, but the `%q` should properly escape any escape sequences, since `root_object_name` and `root_object_query` are user-defined

docs/errors.md

@@ -0,0 +1,27 @@
+# Interface
+
+The `bobject` command generally allows you to 'set' and 'get' different types of variables from some object hierarchy (think JSON object)
+
+## `bobject get-*`
+
+Get operations have two mutually exclusive modes, specified with either `--ref` or `--value` as flags. If you are not sure which to use, pass `--value`
+
+### ref
+
+In ref mode, `REPLY` is a string that is set with the value of the global variable that contains the content you queried for. This functionality is not yet implemented
+
+### value
+
+In value mode, the `REPLY` contains the actual value you want
+
+## `bobject-set-*`
+
+Set operations have two mutually exclusive modes, specified with either `--ref` or `--value` as flags. If you are not sure which to use, pass `--ref`
+
+### ref
+
+In ref mode, a third argument is passed, which is a string that is set to the name of a variable. When setting a field in the object hierarchy, the contents of that particular variable is copied into the hierarchy
+
+### value
+
+In value mode, there is no third argument. After supplying the first two arguments, add `--`, then supply whatever value you want to be used when setting a field in the object hierarchy. This works with variable types `string`, `array`, and `object`

docs/getting-started-technical.md

@@ -0,0 +1,5 @@
+# Virtual Object
+
+Virtual objects are implemented as a string that contains some metadata and a reference to a variable. For example, in `$'\x1C\x1Dtype=array;&SUB_OBJECT'`, the metadata is `type=array`, and the reference is to the (global) variable `SUB_OBJECT`. Virutal objects always start with `$'\x1C\x1D`, to identify them amongst regular strings
+
+The `type=<type>` may look redundant since the type can be checked without subshell creation with `${var@a}` or `${var@A}`. However, this only works on newer Bash versions. Additionally, it is faster to check the types when setting the object and then saving that data as a string, rather than re-checking the type on every access. Lastly, `bash-object` may be extended in the future to support custom objects, so a field for it in the meatdata is useful

docs/interface.md

@@ -1,10 +1,12 @@
 # Query
 
-Queries are modeled from `jq`'s querytree mechanism. The featureset compared to `jq` is significantly less
+Queries are similar to `jq`'s filter mechanism
 
-Both quering modes listed below are are mutually exclusive. By default, the 'simple' method is used by defualt; only if your query contains a `[` will the 'advanced' method be used
+Both quering modes listed below are are mutually exclusive. By default, the 'simple' method is used by default. The parser will automatically switch to 'advanced' mode if your query contains a `[`
 
-# Simple query
+A word of caution, remember there are three different quoting behaviors within Bash itself (`"string"`, `'string'`, `$'string'`), some of which have escape sequences as well. I recommend surrounding queries with `''` where possible
+
+## Simple query
 
 This mode is meant to be fast, and simply splits the dot deliminated string into an array, skipping any empty elements
 
@@ -20,14 +22,17 @@ query='..m!y_k@e y.....su\\]b_"ke\y'
 # => ('m!y_k@e y' 'su\\]b_"ke\y')
 ```
 
-Obviously, writing the query in that way is undesirable, so like `jq`, please ensure keys are made up of alphanumeric characters and underscore (while not starting with a digit)
+Obviously, writing the query in that way is highly undesirable, so like `jq`, _please_ ensure keys are made up of alphanumeric characters and underscores (without starting with a digit)
+
+## Advanced query
 
-# Advanced query
+In this mode, all query elements are contained within square brackets. You must use this syntax if you wish to obtain a particular index from an array
 
-In this mode, query elements are contained within square brackets. You must use this syntax if you wish to obtain a particular index from an array
+The rules follow
 
 - Use quotes for strings (accessing keys in associative arrays)
   - Everything inside quotes is literal, excluding three characters that have escape sequences: `\\`, `\"`, and `\]`
+  - The string _cannot_ begin with `$'\x1C'`, as that is how `bash-object` differentiates between a string and a number
 - Do not use quotes for numbers (accessing indexes in indexed arrays)
 
 ```sh
@@ -38,6 +43,10 @@ query='.["my_key"].[3]'
 # => ('my_key' $'\x1C3')
 ```
 
-A word of caution, remember there are three different quoting behaviors (`"string"`, `'string'`, `$'string'`), some of which have escape sequences as well. I recommend surrounding queries with `''`
+## Invalid queries
+
+The following query is invalid because it mixes the 'simple' and 'advanced' query types within the same query
 
-Lastly, the prepended `$'\x1C'` exists so `bash-object` knows the key is a number without having to re-check it. This might be removed at a later point since it's not required - it's an implementation detail so you don't have to worry about it
+```sh
+query='.my_key.[3]'
+```

docs/internals/virtual-object.md

@@ -4,4 +4,8 @@
 
 ### `TRACE_BASH_OBJECT_TRAVERSE`
 
+Setting this prints diagnostic info for debugging how `bash-object` traverses the object hierarchy
+
 ### `TRACE_BASH_OBJECT_PARSE`
+
+Setting this prints diagnostic info for debugging the querytree parser

docs/query.md

@@ -68,7 +68,7 @@ bash_object.traverse-get() {
 	local -n current_object="$root_object_name"
 
 	# A stack of all the evaluated querytree elements
-	local -a querytree_stack=()
+	# local -a querytree_stack=()
 
 	# Parse the querytree, and recurse over their elements
 	case "$querytree" in
@@ -84,7 +84,7 @@ bash_object.traverse-get() {
 			is_index_of_array='yes'
 		fi
 
-		querytree_stack+=("$key")
+		# querytree_stack+=("$key")
 		# bash_object.util.generate_querytree_stack_string
 		# local querytree_stack_string="$REPLY"
 
@@ -108,9 +108,7 @@ bash_object.traverse-get() {
 
 				virtual_item="${key_value#??}"
 				bash_object.parse_virtual_object "$virtual_item"
-				# shellcheck disable=SC2153
 				local current_object_name="$REPLY1"
-				# shellcheck disable=SC2153
 				local vmd_dtype="$REPLY2"
 				local -n current_object="$current_object_name"
 
@@ -179,7 +177,7 @@ bash_object.traverse-get() {
 							fi
 							;;
 						array)
-							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for object, but found existing array'
+							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
 							return
 							;;
 						*)
@@ -190,7 +188,7 @@ bash_object.traverse-get() {
 					elif [ "$final_value_type" = array ]; then
 						case "$vmd_dtype" in
 						object)
-							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for array, but found existing object'
+							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
 							return
 							;;
 						array)
@@ -214,11 +212,11 @@ bash_object.traverse-get() {
 					elif [ "$final_value_type" = string ]; then
 						case "$vmd_dtype" in
 						object)
-							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for string, but found existing object'
+							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
 							return
 							;;
 						array)
-							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for string, but found existing array'
+							bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
 							return
 							;;
 						*)