You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: book/src/formality_core/parse.md
+31-33Lines changed: 31 additions & 33 deletions
Original file line number
Diff line number
Diff line change
@@ -23,8 +23,8 @@ struct MyEnum {
23
23
24
24
When parsing an enum there will be multiple possibilities. We will attempt to parse them all. If more than one succeeds, the parser will attempt to resolve the ambiguity by looking for the **longest match**. However, we don't just consider the number of characters, we look for a **reduction prefix**:
25
25
26
-
* When parsing, we track the list of things we had to parse. If there are two variants at the same precedence level, but one of them had to parse strictly more things than the other and in the same way, we'll prefer the longer one. So for example if one variant parsed a `Ty` and the other parsed a `Ty Ty`, we'd take the `Ty Ty`.
27
-
* When considering whether a reduction is "significant", we take casts into account. See `ActiveVariant::mark_as_cast_variant` for a more detailed explanation and set of examples.
26
+
- When parsing, we track the list of things we had to parse. If there are two variants at the same precedence level, but one of them had to parse strictly more things than the other and in the same way, we'll prefer the longer one. So for example if one variant parsed a `Ty` and the other parsed a `Ty Ty`, we'd take the `Ty Ty`.
27
+
- When considering whether a reduction is "significant", we take casts into account. See `ActiveVariant::mark_as_cast_variant` for a more detailed explanation and set of examples.
28
28
29
29
### Precedence and left-recursive grammars
30
30
@@ -36,42 +36,41 @@ We support left-recursive grammars like this one from the `parse-torture-tests`:
36
36
37
37
We also support ambiguous grammars. For example, you can code up arithmetic expressions like this:
When specifying the `#[precedence]` of a variant, the default is left-associativity, which can be written more explicitly as `#[precedence(L, left)]`. If you prefer, you can specify right-associativity (`#[precedence(L, right)]`) or non-associativity `#[precedence(L, none)]`. This affects how things of the same level are parsed:
45
44
46
-
*`1 + 1 + 1` when left-associative is `(1 + 1) + 1`
47
-
*`1 + 1 + 1` when right-associative is `1 + (1 + 1)`
48
-
*`1 + 1 + 1` when none-associative is an error.
45
+
-`1 + 1 + 1` when left-associative is `(1 + 1) + 1`
46
+
-`1 + 1 + 1` when right-associative is `1 + (1 + 1)`
47
+
-`1 + 1 + 1` when none-associative is an error.
49
48
50
49
### Symbols
51
50
52
-
A grammar consists of a series of *symbols*. Each symbol matches some text in the input string. Symbols come in two varieties:
53
-
54
-
* Most things are *terminals* or *tokens*: this means they just match themselves:
55
-
* For example, the `*` in `#[grammar($v0 * $v1)]` is a terminal, and it means to parse a `*` from the input.
56
-
* Delimeters are accepted but must be matched, e.g., `( /* tokens */ )` or `[ /* tokens */ ]`.
57
-
* Things beginning with `$` are *nonterminals* -- they parse the contents of a field. The grammar for a field is generally determined from its type.
58
-
* If fields have names, then `$field` should name the field.
59
-
* For position fields (e.g., the T and U in `Mul(Expr, Expr)`), use `$v0`, `$v1`, etc.
60
-
* Exception: `$$` is treated as the terminal `'$'`.
61
-
* Nonterminals have various modes:
62
-
*`$field` -- just parse the field's type
63
-
*`$*field` -- the field must be a `Vec<T>` -- parse any number of `T` instances. Something like `[ $*field ]` would parse `[f1 f2 f3]`, assuming `f1`, `f2`, and `f3` are valid values for `field`.
64
-
*`$,field` -- similar to the above, but uses a comma separated list (with optional trailing comma). So `[ $,field ]` will parse something like `[f1, f2, f3]`.
65
-
*`$?field` -- will parse `field` and use `Default::default()` value if not present.
66
-
*`$<field>` -- parse `<E1, E2, E3>`, where `field: Vec<E>`
67
-
*`$<?field>` -- parse `<E1, E2, E3>`, where `field: Vec<E>`, but accept empty string as empty vector
68
-
*`$(field)` -- parse `(E1, E2, E3)`, where `field: Vec<E>`
69
-
*`$(?field)` -- parse `(E1, E2, E3)`, where `field: Vec<E>`, but accept empty string as empty vector
70
-
*`$[field]` -- parse `[E1, E2, E3]`, where `field: Vec<E>`
71
-
*`$[?field]` -- parse `[E1, E2, E3]`, where `field: Vec<E>`, but accept empty string as empty vector
72
-
*`${field}` -- parse `{E1, E2, E3}`, where `field: Vec<E>`
73
-
*`${?field}` -- parse `{E1, E2, E3}`, where `field: Vec<E>`, but accept empty string as empty vector
74
-
*`$:guard <nonterminal>` -- parses `<nonterminal>` but only if the keyword `guard` is present. For example, `$:where $,where_clauses` would parse `where WhereClause1, WhereClause2, WhereClause3` but would also accept nothing (in which case, you would get an empty vector).
51
+
A grammar consists of a series of _symbols_. Each symbol matches some text in the input string. Symbols come in two varieties:
52
+
53
+
- Most things are _terminals_ or _tokens_: this means they just match themselves:
54
+
- For example, the `*` in `#[grammar($v0 * $v1)]` is a terminal, and it means to parse a `*` from the input.
55
+
- Delimeters are accepted but must be matched, e.g., `( /* tokens */ )` or `[ /* tokens */ ]`.
56
+
- Things beginning with `$` are _nonterminals_ -- they parse the contents of a field. The grammar for a field is generally determined from its type.
57
+
- If fields have names, then `$field` should name the field.
58
+
- For position fields (e.g., the T and U in `Mul(Expr, Expr)`), use `$v0`, `$v1`, etc.
59
+
- Exception: `$$` is treated as the terminal `'$'`.
60
+
- Nonterminals have various modes:
61
+
-`$field` -- just parse the field's type
62
+
-`$*field` -- the field must be a collection of `T` (e.g., `Vec<T>`, `Set<T>`) -- parse any number of `T` instances. Something like `[ $*field ]` would parse `[f1 f2 f3]`, assuming `f1`, `f2`, and `f3` are valid values for `field`.
63
+
-`$,field` -- similar to the above, but uses a comma separated list (with optional trailing comma). So `[ $,field ]` will parse something like `[f1, f2, f3]`.
64
+
-`$?field` -- will parse `field` and use `Default::default()` value if not present.
65
+
-`$<field>` -- parse `<E1, E2, E3>`, where `field` is a collection of `E`
66
+
-`$<?field>` -- parse `<E1, E2, E3>`, where `field` is a collection of `E`, but accept empty string as empty vector
67
+
-`$(field)` -- parse `(E1, E2, E3)`, where `field` is a collection of `E`
68
+
-`$(?field)` -- parse `(E1, E2, E3)`, where `field` is a collection of `E`, but accept empty string as empty vector
69
+
-`$[field]` -- parse `[E1, E2, E3]`, where `field` is a collection of `E`
70
+
-`$[?field]` -- parse `[E1, E2, E3]`, where `field` is a collection of `E`, but accept empty string as empty vector
71
+
-`${field}` -- parse `{E1, E2, E3}`, where `field` is a collection of `E`
72
+
-`${?field}` -- parse `{E1, E2, E3}`, where `field` is a collection of `E`, but accept empty string as empty vector
73
+
-`$:guard <nonterminal>` -- parses `<nonterminal>` but only if the keyword `guard` is present. For example, `$:where $,where_clauses` would parse `where WhereClause1, WhereClause2, WhereClause3` but would also accept nothing (in which case, you would get an empty vector).
75
74
76
75
### Greediness
77
76
@@ -81,8 +80,8 @@ Parsing is generally greedy. So `$*x` and `$,x`, for example, consume as many en
81
80
82
81
If no grammar is supplied, the default grammar is determined as follows:
83
82
84
-
* If a `#[cast]` or `#[variable]` annotation is present, then the default grammar is just `$v0`.
85
-
* Otherwise, the default grammar is the name of the type (for structs) or variant (for enums), followed by `()`, with the values for the fields in order. So `Mul(Expr, Expr)` would have a default grammar `mul($v0, $v1)`.
83
+
- If a `#[cast]` or `#[variable]` annotation is present, then the default grammar is just `$v0`.
84
+
- Otherwise, the default grammar is the name of the type (for structs) or variant (for enums), followed by `()`, with the values for the fields in order. So `Mul(Expr, Expr)` would have a default grammar `mul($v0, $v1)`.
86
85
87
86
### Customizing the parse
88
87
@@ -96,7 +95,6 @@ You must then supply an impl of `Parse` yourself. Because `Parse` is a trait ali
96
95
97
96
In the Rust code, the impl for `RigidTy` looks as follows:
0 commit comments