Running into message serialization failures with SequentialOrchestration

[smokhasi]

Trying to replace an existing AgentGroupChat setup with SequentialOrchestration as per this example

However, I am running into serialization errors when trying to pass messages from one agent to the next when using SequentialOrchestration.

For reference, the agents are created using AzureAssistantAgent and are configured to output json and I run into the below errors with and without structured outputs.

INFO:in_process_runtime.events:{"payload": "Message could not be serialized", "sender": "database_agent_73a7818021d14e5c8a57461143939473/default", "receiver": "research_agent_73a7818021d14e5c8a57461143939473/default", "kind": "MessageKind.DIRECT", "delivery_stage": "DeliveryStage.SEND", "type": "Message"}
INFO:in_process_runtime:Sending message of type SequentialRequestMessage to research_agent_73a7818021d14e5c8a57461143939473: {'body': ChatMessageContent(inner_content=None, ai_model_id=None, metadata={'created_at': 1748924939, 'message_id': 'msg_NTmnPBeTyU9MW2WaTJUuzpVi', 'step_id': 'step_t7nHSjkoBx4WhDmQKBuyvUZf', 'run_id': 'run_Zm2ubDHionJzzJvGLurRGSZt', 'thread_id': 'thread_F2R7WcnxKBqe3Cj29nUrYX30', 'assistant_id': 'asst_clpLSDEJzCnCv2E5jzuMOfSo', 'usage': Usage(completion_tokens=199, prompt_tokens=911, total_tokens=1110, prompt_token_details={'cached_tokens': 0})}, content_type='message', role=<AuthorRole.ASSISTANT: 'assistant'>, name='database_agent', items=[TextContent(inner_content=None, ai_model_id=None, metadata={}, content_type='text', text='{\n  "PROJECT_ID": "project",\n  "ANNUAL": "",\n  "CITY": "",\n  "COUNTRY": "",\n  "COUNTY": "",\n  "OWNER": null,\n  "TYPE": "",\n  "DATA": "-300.0",\n  "DATA3": "",\n  "DATA2": "; Works",\n  "NAME": "Cluster",\n  "TYPE": "Based",\n  "START_DATE": "2025-07-17",\n  "STATE": "",\n  "STATUS": "",\n  "MM": ""\n}', encoding=None)], encoding=None, finish_reason=None, status=None)}
INFO:in_process_runtime:Calling message handler for research_agent_73a7818021d14e5c8a57461143939473/default with message type SequentialRequestMessage sent by database_agent_73a7818021d14e5c8a57461143939473/default

Are there any best practices/recommendations which need to be followed when passing around json objects from one agent to the next? Should they follow a particular schema and if so, should the schema be provided as attribute anywhere?

For reference, we have multiple agents which are instructed to generated their output as a json object when not using structured output and it works well with AgentGroupChat

[smokhasi]

For additional context,
I believe the issue is agent_response_callback is only an observer function and does not allow to manipulate the object before passing the message onto the next agent and from the looks of it, the agents are expecting an object of type ChatMessageContent but do not understand how to serialize pydantic base models into ChatMessageContent.

If the function agent_response_callback can be modified to return values similar to input_transform, we can provide something like

def custom_agent_transform(input_message: BaseModel) -> ChatMessageContent:
    # convert input message to a chatMessage content
    return ChatMessageContent(role=AuthorRole.ASSISTANT, content=input_message.model_dump_json())

[moonbox3]

Adding @TaoChenOSU

[TaoChenOSU]

Hey @smokhasi,

Thanks for reporting the issue!

Internally, the agents communicate through ChatMessageContent regardless of if structured output is enabled. If you look at this sample here, you can see that the structured output is actually just a JSON string, and the developer needs to deserialize it into the object.

Just by looking at the logs you posted, we are not able to come up with a hypothesis as to what went wrong. Is it possible for you to provide more details of your agent setup?

[smokhasi]

Thank you. Looks like the issue was an older data plane inference api version.
Resolved after updating to 2025-04-01-preview