Add Automatic Context Summarization to ClientSession

[mldk-tech]

Summary

Adds the ClientSessionSummarizing class, extending ClientSession to provide automatic summarization of conversation context, preventing token overflow in long-running sessions. Introduces precise token counting with the tiktoken library and provides flexible configuration options.

Motivation and Context

In extended conversations, maintaining large message histories can lead to exceeding token limits, resulting in potential session interruptions or model failures. This feature ensures that conversations are automatically summarized once a defined token threshold is reached, ensuring smooth and uninterrupted operation.

How Has This Been Tested?

• Unit Tests: Added 7 unit tests covering scenarios like:

  • Approaching token limits
  • Customization of parameters (max_tokens, summarize_threshold, summary_prompt)
  • Ensuring summarization functionality triggers correctly

• Local Testing: All tests have passed successfully in the local environment with:

  • Ruff format and checks ✅
  • Pyright type checks ✅
  • Pytest unit tests ✅

Breaking Changes

No breaking changes. Users will not be forced to update existing configurations or code.
However, a new optional dependency (tiktoken) has been added to enable precise token management.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the [MCP Documentation](https://modelcontextprotocol.io)
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

• New Dependency: tiktoken for accurate token counting, ensuring session stability.

• Configurable Parameters:

  • max_tokens (default: 4000 tokens)
  • summarize_threshold (default: 0.8 – 80%)
  • summary_prompt (default provided, user customizable)

This implementation is production-ready and provides flexibility and robust performance for users managing extended interactions with language models.

[Kludex]

Sorry, but I don't think this fits in the SDK.

But feel free to create your own package with it.