Documentation request: How to consume Token that uses groups?

[1NF053C]

The lexer supports groups, as shown in this example here: https://github.com/Chevrotain/chevrotain/blob/master/examples/lexer/token_groups/token_group_spec.js#L24

Are there docs that demonstrate usage of $.CONSUME(MyTokenWithGroups) in the Parser grammar?

[msujew]

@1NF053C As far as I know, groups aren't supposed to be consumed by the parser at all, as they aren't part of the normal token stream that gets returned by the Lexer class. They are intended for tokens that live outside of the parser, such as whitespace tokens or comments.

Are you perhaps confusing groups with token categories?

[1NF053C]

@msujew not sure 🙂 I was more or less able to accomplish what I needed with $.RULE, $.MANY and $.OR

Categories may have helped me simplify some things but I’m not sure after taking a quick glance at those docs.

I had to process a few simple tokens in some unknown order within a longer string while retaining all of the stuff in between so I created some tokens with alternative negative lookaheads to make this sort of work