feat: add new generic AI communication model

[sdirix]

What it does

Adds a new AI communication model for tracking raw LLM requests and
responses. Tracking is automated via the language-model-service.

This model serves as the foundation for:

• Extracting communication history from Theia AI, allowing LLM testing
  without running the full Theia-based application
• Replaying LLM communication without using an actual LLM, enabling
  deterministic integration tests
• Removing the clunky communication recording service, making the
  ai-history package optional

Resolves #15221

Remaining tasks:

• Integrate feedback regarding naming and recording of the communication model
• Adapt History View UI
  • Track timestamps and response timestamps differently
  • Do not highlight all semantic requests
  • Show logged streamed responses in a nicer UI
• Remove communication service, adapt all affected code and make ai-history package optional
  • Update changelog for breaking change

Moved the remaining tasks into a new PR (#15540) to make the PRs smaller.

How to test

Debug the language-model-service.

Alternatively you can check the follow up PR (#15540) to see the new model integrated in the AI History view.

Follow-ups

Breaking changes

• This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Contributed on behalf of STMicroelectronics

Review checklist

• As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• As a reviewer, I agree to behave in accordance with the review guidelines

[planger]

Thank you very much Stefan!

This looks already very good and it is great to get rid of the clunky manual recording and this should be a solid foundation to extract all sorts of useful data for testing, simulation, etc.

I was wondering how we reflect function calls? I don't see them being recorded at the moment, right?

The remaining comments I'd have are related to naming:

AiSession, AiSemanticRequest, AiRequest

I'm not sure "Ai" is a great prefix. Essentially it is a record of a LanguageModelRequest or session. Also I find the use of the term "Semantic" a bit unclear.

Here are a few ideas, instead of switching to a different prefix, like Ai, and avoiding "semantic":

• AiSession -> RawLanguageModelSession or LanguageModelInteraction
• AiSemanticRequest -> ClientRequest (the original request for tracing raw language model requests to a client event)
• AiRequest -> RawLanguageModelRequest

Thank you!

[sdirix]

I was wondering how we reflect function calls? I don't see them being recorded at the moment, right?

Tool Calls including their ids, parameters and result are recorded as part of the responses, see here.

The remaining comments I'd have are related to naming:

AiSession, AiSemanticRequest, AiRequest

I'm not sure "Ai" is a great prefix. Essentially it is a record of a LanguageModelRequest or session. Also I find the use of the term "Semantic" a bit unclear.

Here are a few ideas, instead of switching to a different prefix, like Ai, and avoiding "semantic":

• AiSession -> RawLanguageModelSession or LanguageModelInteraction
• AiSemanticRequest -> ClientRequest (the original request for tracing raw language model requests to a client event)
• AiRequest -> RawLanguageModelRequest

With semantic request I wanted to express that it's representing what the user might think of a single request. I'll further think about the naming.

Thanks for the feedback!

[planger]

With semantic request I wanted to express that it's representing what the user might think of a single request. I'll further think about the naming.

Thanks! Yes that makes a lot of sense to capture that. It could also be any client, such as a service though, right? Therefore I was leaning towards ClientRequest, but I'm really open to anything. I can also live with "semantic", but this term is a bit convoluted to me. :-)

[sdirix]

Hi @planger, Thanks for your input!

I adapted the naming a bit and split the PR into two PRs:

• This PR introducing the new model
• The follow up PR (refactor: use communication model in AI history #15540) in which the new model is used to refactor the ai-history package

Please check whether you like it.

[planger]

Excellent, thank you! This looks very clean and useful to me. I tested it with #15540

[planger]

Just a nitpick: I am ok with LanguageModelExchange but I was wondering what you think about LanguageModelCompletion as a more direct term for what actually happens (i.e. an LLM request completion). Exchange sounds to me a bit bi-directional and not a perfect fit.

[sdirix]

The exchange consists of request-response pairs, so it feels bidirectional to me. Personally I like "exchange" better than "completion". I would have never guessed that a "LanguageModelCompletion" is a set of request-response pairs, unrelated to "normal" LLM completion.