Add Support for # Comments in Think Language

[lwgray]

Overview

Add support for single-line comments in Think using the '#' symbol, similar to Python's comment syntax. This will allow users to add explanatory notes and documentation within their Think programs.

Current State

Currently, Think has no support for comments. Users cannot add explanatory text or disable code without removing it.

Proposed Change

Add support for '#' style comments that:

1. Can appear on their own line
2. Can appear at the end of code lines
3. Extend from the '#' character to the end of the line

Example Usage

# This is a full-line comment
objective "Calculate student grades"  # This is an end-of-line comment

task "Process Grades":  # Process student examination scores
    # Initialize variables
    step "Setup":
        scores = [85, 92, 78]  # Raw test scores

Implementation Details

1. Lexer Changes (parser.py)

Add a new token rule to the lexer that ignores everything from '#' to the end of the line:

def t_COMMENT(t):
    r'\#.*'
    pass  # Comments should be ignored by the lexer

This rule should be added to the ThinkParser class before the IDENTIFIER rule to ensure proper precedence.

2. Update Tests (test_parser.py)

Add new test cases to verify comment handling:

def test_comments(self, parser):
    """Test parsing of comments in various positions."""
    code = '''# Full line comment
    objective "Test"  # End of line comment
    task "Example":  # Task comment
        step "Do Something":  # Step comment
            x = 42  # Variable assignment comment
    run "Example"  # Final comment'''
    
    ast = parser.parse(code)
    assert ast['objective'] == 'Test'
    assert len(ast['tasks']) == 1
    assert ast['tasks'][0]['name'] == 'Example'

3. Documentation Updates

1. Update Think Language Style Guide (PEP 1)
2. Add examples to README.md
3. Update Jupyter notebook examples
4. Update educational materials

4. Error Handling Considerations

1. Ensure comments are properly handled in error messages and line number reporting
2. Update the source context display in error messages to show comments
3. Verify that comments don't interfere with indentation parsing

Migration Impact

This is a backward-compatible change. Existing Think programs will continue to work as before.

Testing Plan

1. Basic Comment Tests:

   • Full-line comments
   • End-of-line comments
   • Multiple consecutive comment lines
   • Comments with special characters
   • Unicode characters in comments

2. Integration Tests:

   • Comments in all program sections (objective, task, step, etc.)
   • Comments with indentation
   • Comments in control structures (if/else, loops)
   • Comments in error scenarios

3. Edge Cases:

   • Empty comments (#)
   • Comments at file start/end
   • Multiple #'s in one line
   • Comments with quotes/brackets
   • Very long comments

Acceptance Criteria

1. Parser correctly ignores comments during execution
2. Error messages show correct line numbers with comments present
3. Source context in error messages includes comments
4. All existing tests pass
5. Documentation is updated
6. Examples include comment usage
7. No performance degradation in parsing

Related Documentation

• Current parser implementation: parser.py
• Test suite: test_parser.py
• Style guide: PEP 1 - ThinkPy Style Guide.md