Fix chunking bug when newlines fall at chunk boundaries (#71)

[aksg87]

Description

Fix chunking bug that created empty intervals when newlines fall at chunk boundaries.

The ChunkIterator was creating empty TokenInterval objects (start_index == end_index) when newlines coincided with chunk boundaries, causing a ValueError. Added check to ensure intervals are non-empty.

Fixes #71

Bug fix

How Has This Been Tested?

$ python -m pytest tests/chunking_test.py -v
# All 18 tests pass including new regression test
$ python -m pytest tests/
# All 161 tests pass

Checklist:

• I have read and acknowledged Google's Open Source Code of conduct.
• I have read the Contributing page, and I either signed the Google Individual CLA or am covered by my company's Corporate CLA.
• I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
• I have made any needed documentation changes, or noted in the linked issue(s) that documentation elsewhere needs updating.
• I have added tests, or I have ensured existing tests cover the changes
• I have followed Google's Python Style Guide and ran pylint over the affected code.

[github-actions]

⚠️ Branch Update Required

Your branch is 17 commits behind main. Please update your branch to ensure CI checks run with the latest code:

git fetch origin main
git merge origin/main
git push

Note: Enable "Allow edits by maintainers" to allow automatic updates.