- 
                Notifications
    You must be signed in to change notification settings 
- Fork 486
[Feature] Enable list[dict] type for message['content'] for two-agent chat and group chat APIs #2134
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
| @Lancetnik would you be able to look over this for compatibility with remote agent. | 
| @marklysze it should works fine as I can see, but I am not sure, that is a best option | 
| the following test are failing due image_url not being handled in openai_responses.py: I will be adding fixes here shortly along with backward compatibility. | 
fix: broken test
426bf63    to
    d7429aa      
    Compare
  
    | 
 @priyansh4320 How did you test? I can't reproduce with  | 
| 
 ./scripts/test-core-skip-llm.sh | 
| 
 | 
| 
 @priyansh4320 I ran all test from "Update e2e test" test-core-skip-llm.sh is not even the right way to test this. The integration test is supposed to be triggered with LLM If I ran I got  | 
| @randombet the failure should be reproducable with   | 
| Not sure what do you mean. And what are you trying to fix with additional commits? @priyansh4320 | 
| Create #2145 instead | 
Why are these changes needed?
OAI Response API and new Gemini client support list[dict] in their output content field. See below for details.
OAI Response API output is a list of ResponseOutputItem. E.g. output of an image generation could be a list of two items, one text output and one encoded image string.
To support use cases like this, I would like to make AG2 message to messages.
Similarly, the content of OAI response API input (https://github.com/openai/openai-python/blob/main/src/openai/types/responses/response_input_item.py#L42 and https://github.com/openai/openai-python/blob/main/src/openai/types/responses/easy_input_message.py#L13) is a ResponseInputMessageContentList
Gemini Chat also support sending a list of messages https://github.com/googleapis/python-genai/blob/main/google/genai/chats.py#L227
Related issue number
Checks