BTEQ - ignoring dot in front of command

[MarkBell920]

I am trying to extend the Teradata dialect to parse and ignore BTEQ commands. I have manged to add the common commands to the teradta.py file but the problem is BTEQ commands do not have to start with a dot (.). They are only 'Optional, but recommended' in the Teradata documentation. I know the SQL I am trying to parse will have a mix of commands with a dot in front and commands without.

Does anyone have an example of how I can make the dot optional in commands like .LOGON/LOGON?

thanks,

Mark

[georgesittas]

Perhaps you could override the tokenizer to handle a possible DOT token as a prefix to a COMMAND? Not sure if that's correct in all cases. Can you show an example of what you're trying to achieve?

[MarkBell920]

I want to catch both of these lines in a Teradata script

.SET SIDETITLES ON;

SET SIDETITLES ON;

Both commands are valid.

[georgesittas]

If you just want to parse them into Commands, i.e. instead of fully parsing them just store them as raw strings, you could create a mapping of strings that represent the starting keywords for each command of interest into TokenType.COMMAND, and use that mapping to also add their dot-prefixed variants with a simple comprehension in the Tokenizer.KEYWORDS dict.

For example:

class Tokenizer:
    ...
    DOT_PREFIXED_COMMANDS = {
        "SET": TokenType.COMMAND,
        ...
    }

    KEYWORDS = {
        ...
        **DOT_PREFIXED_COMMANDS,
        **{f".{command}": TokenType.COMMAND for command in DOT_PREFIXED_COMMANDS},
    }