Reducing coupling between OpenAIChatGenerator and core integrations #9769
Description
Status
Currently, 6 core integrations inherit from the OpenAIChatGenerator, since their APIs are OpenAI-compatible.
This brings some benefits: for example, when we introduced image support in OpenAI, all 6 integrations (+ Azure) automatically gained that feature.
However, this approach also causes problems:
-
Fragility with upstream changes: When OpenAI changed their API to support custom tools, we made a fix to avoid errors in OpenAI. After Haystack release, we discovered it broke Mistral (undiscoverable with unit tests). This required extra work and a bugfix release of Haystack (see chore: OpenAI - explictly filter function tool calls #9697 and fix:
OpenAIChatGenerator- improve the logic to exclude custom tool calls #9725) -
Constraints on new OpenAI features: When implementing structured outputs in OpenAI (feat: support structured outputs in
OpenAIChatGenerator#9754), we had to avoid a design that was more natural for OpenAI but risked breaking dependencies. -
Testing: Even with unit tests in Haystack main, we are not always able to do an early detection of downstream bugs we introduce by changing OpenAI.
Proposal
To better handle this, I suggest:
- Introduce a minimal
OpenAIBasicChatGenerator, for internal usage only, not documented or promoted - This would implement just the generic superset of features we typically need in integrations.
- (if possible, design it in a modular way, so specific behaviors can easily be overridden when necessary).
- Place it in core-integrations, making it easier to release it and to test (potentially with integrations depending on it).
In the meantime, a mitigation would be to nightly run also integration tests with Haystack for integrations inheriting from the OpenAIChatGenerator.
This does not solve the problem, but would allow us to catch bugs earlier.
I would be happy to discuss these ideas or explore better alternatives.