Parsing with alternatives, how to "fall back"?

[totalgee]

I'm missing something basic here to do with choice... The following code works fine for input that is a complexrange (such as "1,3..10"), but fails for something that should match simplerange ("1..10") or simplestep ("7"). I'm not sure how I need to use peek or lookahead here, to get compound_range to work (so it will parse any of the three representations successfully).

https://lexy.foonathan.net/playground/?id=14zej9o4P&mode=tree

#include <lexy/callback.hpp>

struct StepRange {
    int first{};
    int last{};
    int step{};
};

struct simplestep { // "6"
    static constexpr auto rule = dsl::integer<int>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct simplerange { // "1..10"
    static constexpr auto rule = dsl::integer<int> >> dsl::twice(dsl::period) + dsl::integer<int>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct complexrange { // "1,3..9"
    static constexpr auto rule = dsl::integer<int> >> dsl::comma + dsl::integer<int> >> dsl::twice(dsl::period) + dsl::integer<int>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct compound_range {
    static constexpr auto rule = dsl::p<complexrange>
        | dsl::p<simplerange>
        | dsl::p<simplestep>
        ;
    static constexpr auto value = lexy::forward<StepRange>;
};

Thanks for any suggestions on the best way to do this...I think I'm very close...

[totalgee]

With the above example, this parses correctly: 1,3..9

This fails: 1..10

error: while parsing complexrange
     |
   1 | 1..10
     |  ^ expected ','

This also fails: 7

error: while parsing complexrange
     |
   1 | 7
     |  ^ expected ','

Obviously, it's not switching to try the other choice options... I thought if the first one failed, it would then try the next. In fact, I've done some of my own "command parsing" examples which use choice (also, the simple hex parser example in the docs), and they worked as I expected.

[totalgee]

Okay, using dsl::peek I got it working; let me know if you think there's a better way to do it.

https://lexy.foonathan.net/playground/?id=f5x5xY58T&mode=tree

The modified lines are shown like this // <--- using peek

#include <lexy/callback.hpp>

struct StepRange {
    int first{};
    int last{};
    int step{};
};

struct simplestep { // "6"
    static constexpr auto rule = dsl::integer<int>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct simplerange { // "1..10"
    static constexpr auto rule = dsl::peek(dsl::while_one(dsl::digit<>) >> dsl::twice(dsl::period)) // <--- using peek
        >> dsl::integer<int16_t>
        >> dsl::twice(dsl::period)
        + dsl::integer<int16_t>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct complexrange { // "1,3..9"
    static constexpr auto rule = dsl::peek(dsl::while_one(dsl::digit<>) + dsl::comma) // <--- using peek
        >> dsl::integer<int16_t>
        >> dsl::comma
        + dsl::integer<int16_t>
        + dsl::twice(dsl::period)
        + dsl::integer<int16_t>;
    static constexpr auto value = lexy::construct<StepRange>;
};

struct compound_range {
    static constexpr auto rule = dsl::p<complexrange>
        | dsl::p<simplerange>
        | dsl::p<simplestep>
        ;
    static constexpr auto value = lexy::forward<StepRange>;
};

[foonathan]

Yes, dsl::peek is the way you achieve the branching behavior you want. lexy does not do backtracking unless you opt-in to it, as explained here: https://lexy.foonathan.net/learn/branching/

[foonathan]

You can also avoid dsl::peek by merging the grammar and parsing the optional suffixes: https://lexy.foonathan.net/playground/?id=czKGez4nM&mode=tree

But note that when constructing a complex range, this (and your solution), will fill something like 1,2..3 with {.first = 1, .last = 2, .step = 3} which is probably not what you want. You can fix that by using lexy::as_aggregate: https://lexy.foonathan.net/reference/callback/aggregate/#as_aggregate

[totalgee]

Thanks, that's good to also know about as_aggregate!

Yes, I know my simple example won't actually set the step correctly. The actual class I'm using has a constructor with three ints, which properly calculates and initializes its step field from the "first, second and end" values that are parsed. I just simplified it here because the problem I was having was with the parsing, not the other logic. ;-)

[totalgee]

You can also avoid dsl::peek by merging the grammar and parsing the optional suffixes: https://lexy.foonathan.net/playground/?id=czKGez4nM&mode=tree

What about if I wanted another production range_list that could take a space-separated list of these compound_range elements, and output a std::vector<StepRange>? I managed to do it with my peek approach, but not with your more elegant dsl::if_ approach that doesn't "peek" -- which otherwise I love...

My version (that works with my more complex three-productions compound_range) shown above is something like this (ugly but it works):

        struct range_list {
            static constexpr auto rule = [] {
                return dsl::opt(dsl::peek(dsl::integer<int>) >> dsl::no_whitespace(dsl::list(dsl::p<compound_range>, dsl::sep(dsl::while_one(dsl::ascii::space)))));
            }();
            static constexpr auto value = lexy::as_list<std::vector<StepRange>>;
        };

[foonathan]

You need dsl::p<compound_range> to be a branch for that case, so you can use >> to specify what is used as the branch condition: https://lexy.foonathan.net/playground/?id=vv1E13oxP&mode=tree

[totalgee]

Thanks, I like this much simpler version of compound_range, and I got it to work with my range_list (for my use case, which is a bit more complex than this example). Thanks again for the help!