Skip to content

Commit fdb8819

Browse files
committed
docs: Improve docs content
1 parent 70210f3 commit fdb8819

File tree

12 files changed

+176
-148
lines changed

12 files changed

+176
-148
lines changed

docs/bobject-get.md

Lines changed: 0 additions & 15 deletions
This file was deleted.

docs/bobject.set.md

Lines changed: 0 additions & 11 deletions
This file was deleted.

docs/errors.md

Lines changed: 10 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,21 @@
11
# Errors
22

3-
Because Bash often does the wrong thing silently, it's important to fail early when any inputs are unexpected. Extra checks that require spawning subshells require setting the `VERIFY_BASH_OBJECT` variable. We recommend exporting it so any child Bash processes keep the verification behavior
3+
Because Bash often does the wrong thing silently, it's important to fail early. This page documents the types of errors `bash-object` catches when using its functionality
44

5-
The following checks are performed
5+
Export the `VERIFY_BASH_OBJECT` variable if you want to perform extra validation checks on input. On newer Bash versions this will incur little overhead; on older ones this will spawn subshells (which is why it's not by default enabled)
66

7-
1. Argument parsing
7+
## Error Categories
88

9-
- Ensures the correct number of arguments are passed (when applicable)
10-
- Ensures the arguments are not empty (when applicable)
11-
- Ensures applicable arguments refers to real variables (ex. the root object in both set/get, and the final_value for set)
12-
- Ensure applicable arguments refer to variables of the same type (for set)
9+
Generally, there are three categories of errors
1310

14-
2. Query parsing
11+
### 1. Arguments
1512

16-
- Ensures the query has the correct form
13+
Prefixed with `ERROR_ARGUMENTS_`, these error codes will show when there is something wrong with the passed flags or arguments. We will check to ensure the correct number of arguments have been passed (and are non empty). Additionally, we will ensure that the passed arguments have continuity with whatever you reference. For example, if you want to `--ref set-object`, we will ensure that the variable you specify is a defined variable, and actually is an object (associative array). Another example, if you wish to `set` or `get` a variable, it will check to ensure the variable at the place you specify with the querystring is the same type as you specified as an argument (e.g. `set-array`)
1714

18-
3. Tree traversing
15+
### 2. Querytree
1916

20-
- Ensure type specified in the virtual object matches that of the referenced object
21-
- Ensure type implied in query string matches the one specified in the virtual object
17+
Prefixed with `ERROR_QUERYTREE_`, these error codes will show when there is something wrong with parsing the querytree, namely syntax errors. A message associated with the error code will specify why the querytree couldn't be fixed, along with possible solutions
2218

23-
4. Final operation (set/get)
19+
### 3. Virtual objects
2420

25-
- Ensure type specified in the virtual object matches that of the referenced object
26-
- Ensure type implied in query string matches the one specified in the virtual object
27-
- Ensure type specified in set/get matches the one specified in the virtual object
21+
Prefixed with `ERROR_VOBJ_`, these error codes will show when the virtual object does not match up with its reference. For example, if a virtual object looks like `$'\x1C\x1Dtype=array;&SUB_OBJECT'`, an error will show because it is referencing an object with the variable name `SUB_OBJECT`, but `type=array` is specified

docs/getting-started-technical.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Unmarshaling
2+
3+
Unmarshaling strings representing highly structured data in Bash in a reliable and easy to use way has historically been impossible - until now
4+
5+
The following examples will use JSON format, which represents the data that Bash will encode in-memory
6+
7+
## Motivation
8+
9+
Bash allows for simple/flat mapping via an associative array
10+
11+
```json
12+
{
13+
"atom": "Hydrogen"
14+
}
15+
```
16+
17+
```bash
18+
assert [ "${OBJECT[atom]}" = 'Hydrogen' ]
19+
```
20+
21+
But, when it comes to even the slightest of more complicated structures, Bash cannot cope: associative arrays do not nest inside one another
22+
23+
```json
24+
{
25+
"stars": {
26+
"cool": "Wolf 359"
27+
}
28+
}
29+
```
30+
31+
Note only that, but indexed arrays cannot nest inside associative arrays
32+
33+
```json
34+
{
35+
"nearby": [
36+
"Alpha Centauri A",
37+
"Alpha Centauri B",
38+
"Proxima Centauri",
39+
"Barnard's Star",
40+
"Luhman 16"
41+
]
42+
}
43+
```
44+
45+
## Solution
46+
47+
`bash-object` solves this problem, in the most general case. The repository contains functions for storing and retreiving strings, indexed arrays, and associative arrays in a heterogenous hierarchy
48+
49+
Let's take a look at the most basic case
50+
51+
```json
52+
{
53+
"xray": {
54+
"yankee": "zulu"
55+
}
56+
}
57+
```
58+
59+
In Bash, it will be stored in memory eventually using declarations similar to the following
60+
61+
```sh
62+
declare -A unique_global_variable_xray=([yankee]='zulu')
63+
declare -A OBJECT=([xray]=$'\x1C\x1Dtype=object;&unique_global_variable_xray')
64+
```
65+
66+
You can retrieve the data with
67+
68+
```sh
69+
bobject get-object --value 'OBJECT' 'xray'
70+
assert [ "${REPLY[yankee]}" = zulu ]
71+
```
72+
73+
The implementation hinges on Bash's `declare -n`. When using `get-object`, this is what would happen behind the scenes at the lowest level
74+
75+
```sh
76+
local current_object_name='unique_global_variable_xray'
77+
local -n current_object="$current_object_name"
78+
79+
declare -gA REPLY=()
80+
local key=
81+
for key in "${!current_object[@]}"; do
82+
REPLY["$key"]="${current_object["$key"]}"
83+
done
84+
```
85+
86+
Another implementation detail is how a new variable is created in the global scope. This can occur when you are setting an array (indexed array) or object (associative array) at some place in the object hierarchy. In the previous example, `unique_global_variable_xray` is the new variable created in the global scope; in practice, the name looks a lot different, as seen in the example below
87+
88+
```sh
89+
local global_object_name=
90+
printf -v global_object_name '%q' "__bash_object_${root_object_name}_${root_object_query}_${RANDOM}_${RANDOM}"
91+
92+
if ! eval "declare -gA $global_object_name=()"; then
93+
bash_object.util.die 'ERROR_INTERNAL' 'Eval declare failed'
94+
return
95+
fi
96+
```
97+
98+
Unfortunately, it must use eval, but the `%q` should properly escape any escape sequences, since `root_object_name` and `root_object_query` are user-defined

docs/interface.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# Interface
2+
3+
The `bobject` command generally allows you to 'set' and 'get' different types of variables from some object hierarchy (think JSON object)
4+
5+
## `bobject get-*`
6+
7+
Get operations have two mutually exclusive modes, specified with either `--ref` or `--value` as flags. If you are not sure which to use, pass `--value`
8+
9+
### ref
10+
11+
In ref mode, `REPLY` is a string that is set with the value of the global variable that contains the content you queried for. This functionality is not yet implemented
12+
13+
### value
14+
15+
In value mode, the `REPLY` contains the actual value you want
16+
17+
## `bobject-set-*`
18+
19+
Set operations have two mutually exclusive modes, specified with either `--ref` or `--value` as flags. If you are not sure which to use, pass `--ref`
20+
21+
### ref
22+
23+
In ref mode, a third argument is passed, which is a string that is set to the name of a variable. When setting a field in the object hierarchy, the contents of that particular variable is copied into the hierarchy
24+
25+
### value
26+
27+
In value mode, there is no third argument. After supplying the first two arguments, add `--`, then supply whatever value you want to be used when setting a field in the object hierarchy. This works with variable types `string`, `array`, and `object`

docs/internals/virtual-object.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Virtual Object
2+
3+
Virtual objects are implemented as a string that contains some metadata and a reference to a variable. For example, in `$'\x1C\x1Dtype=array;&SUB_OBJECT'`, the metadata is `type=array`, and the reference is to the (global) variable `SUB_OBJECT`. Virutal objects always start with `$'\x1C\x1D`, to identify them amongst regular strings
4+
5+
The `type=<type>` may look redundant since the type can be checked without subshell creation with `${var@a}` or `${var@A}`. However, this only works on newer Bash versions. Additionally, it is faster to check the types when setting the object and then saving that data as a string, rather than re-checking the type on every access. Lastly, `bash-object` may be extended in the future to support custom objects, so a field for it in the meatdata is useful

docs/query.md

Lines changed: 17 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,12 @@
11
# Query
22

3-
Queries are modeled from `jq`'s querytree mechanism. The featureset compared to `jq` is significantly less
3+
Queries are similar to `jq`'s filter mechanism
44

5-
Both quering modes listed below are are mutually exclusive. By default, the 'simple' method is used by defualt; only if your query contains a `[` will the 'advanced' method be used
5+
Both quering modes listed below are are mutually exclusive. By default, the 'simple' method is used by default. The parser will automatically switch to 'advanced' mode if your query contains a `[`
66

7-
# Simple query
7+
A word of caution, remember there are three different quoting behaviors within Bash itself (`"string"`, `'string'`, `$'string'`), some of which have escape sequences as well. I recommend surrounding queries with `''` where possible
8+
9+
## Simple query
810

911
This mode is meant to be fast, and simply splits the dot deliminated string into an array, skipping any empty elements
1012

@@ -20,14 +22,17 @@ query='..m!y_k@e y.....su\\]b_"ke\y'
2022
# => ('m!y_k@e y' 'su\\]b_"ke\y')
2123
```
2224

23-
Obviously, writing the query in that way is undesirable, so like `jq`, please ensure keys are made up of alphanumeric characters and underscore (while not starting with a digit)
25+
Obviously, writing the query in that way is highly undesirable, so like `jq`, _please_ ensure keys are made up of alphanumeric characters and underscores (without starting with a digit)
26+
27+
## Advanced query
2428

25-
# Advanced query
29+
In this mode, all query elements are contained within square brackets. You must use this syntax if you wish to obtain a particular index from an array
2630

27-
In this mode, query elements are contained within square brackets. You must use this syntax if you wish to obtain a particular index from an array
31+
The rules follow
2832

2933
- Use quotes for strings (accessing keys in associative arrays)
3034
- Everything inside quotes is literal, excluding three characters that have escape sequences: `\\`, `\"`, and `\]`
35+
- The string _cannot_ begin with `$'\x1C'`, as that is how `bash-object` differentiates between a string and a number
3136
- Do not use quotes for numbers (accessing indexes in indexed arrays)
3237

3338
```sh
@@ -38,6 +43,10 @@ query='.["my_key"].[3]'
3843
# => ('my_key' $'\x1C3')
3944
```
4045

41-
A word of caution, remember there are three different quoting behaviors (`"string"`, `'string'`, `$'string'`), some of which have escape sequences as well. I recommend surrounding queries with `''`
46+
## Invalid queries
47+
48+
The following query is invalid because it mixes the 'simple' and 'advanced' query types within the same query
4249

43-
Lastly, the prepended `$'\x1C'` exists so `bash-object` knows the key is a number without having to re-check it. This might be removed at a later point since it's not required - it's an implementation detail so you don't have to worry about it
50+
```sh
51+
query='.my_key.[3]'
52+
```

docs/reference.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,4 +4,8 @@
44

55
### `TRACE_BASH_OBJECT_TRAVERSE`
66

7+
Setting this prints diagnostic info for debugging how `bash-object` traverses the object hierarchy
8+
79
### `TRACE_BASH_OBJECT_PARSE`
10+
11+
Setting this prints diagnostic info for debugging the querytree parser

docs/unmarshaling.md

Lines changed: 0 additions & 80 deletions
This file was deleted.

pkg/lib/traverse-get.sh

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ bash_object.traverse-get() {
6868
local -n current_object="$root_object_name"
6969

7070
# A stack of all the evaluated querytree elements
71-
local -a querytree_stack=()
71+
# local -a querytree_stack=()
7272

7373
# Parse the querytree, and recurse over their elements
7474
case "$querytree" in
@@ -84,7 +84,7 @@ bash_object.traverse-get() {
8484
is_index_of_array='yes'
8585
fi
8686

87-
querytree_stack+=("$key")
87+
# querytree_stack+=("$key")
8888
# bash_object.util.generate_querytree_stack_string
8989
# local querytree_stack_string="$REPLY"
9090

@@ -108,9 +108,7 @@ bash_object.traverse-get() {
108108

109109
virtual_item="${key_value#??}"
110110
bash_object.parse_virtual_object "$virtual_item"
111-
# shellcheck disable=SC2153
112111
local current_object_name="$REPLY1"
113-
# shellcheck disable=SC2153
114112
local vmd_dtype="$REPLY2"
115113
local -n current_object="$current_object_name"
116114

@@ -179,7 +177,7 @@ bash_object.traverse-get() {
179177
fi
180178
;;
181179
array)
182-
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for object, but found existing array'
180+
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
183181
return
184182
;;
185183
*)
@@ -190,7 +188,7 @@ bash_object.traverse-get() {
190188
elif [ "$final_value_type" = array ]; then
191189
case "$vmd_dtype" in
192190
object)
193-
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for array, but found existing object'
191+
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
194192
return
195193
;;
196194
array)
@@ -214,11 +212,11 @@ bash_object.traverse-get() {
214212
elif [ "$final_value_type" = string ]; then
215213
case "$vmd_dtype" in
216214
object)
217-
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for string, but found existing object'
215+
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
218216
return
219217
;;
220218
array)
221-
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' 'Queried for string, but found existing array'
219+
bash_object.util.die 'ERROR_ARGUMENTS_INCORRECT_TYPE' "Queried for string, but found existing $vmd_dtype"
222220
return
223221
;;
224222
*)

0 commit comments

Comments
 (0)