No on_chat_model_stream Event with us.meta.llama3-2-90b-instruct-v1:0 Model from Bedrock

[dpptinh]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

user_input_data = {"messages": [HumanMessage(content=user_input)]}
logger.info(f"Starting stream")
async for event in self.graph.astream_events(
    user_input_data,
    version="v2",
    config=config,
):
    logger.info(f"Event: {event}")

    if "on_chat_model_stream" == event["event"]:
        chunk_content = event["data"]["chunk"].content
        if isinstance(chunk_content, list):
            text_content = ''.join([c['text'] for c in chunk_content if 'text' in c])
        else:
            text_content = chunk_content  
        chunks.append(text_content)
        yield f"event: answer\ndata: {{'v': '{text_content}'}}\n\n"
    if "on_tool_start" == event["event"]:
        yield f"event: tool_start\ndata: {{'v': {event['data']}, 'name': {event['name']}}}\n\n"

    if "on_tool_end" == event["event"]:
        yield f"event: tool_end\ndata: {{'v': {event['data']['output'].content}, 'name': {event['name']}}}\n\n"
# yield f"event: end\ndata: {{'v': {''.join(chunks)}}}\n\n"
# logger.info(f"Final chunk: {''.join(chunks)}")
logger.info(f"Ending stream")
# close the pool
await pool.close()
logger.info(f"Pool closed")

Description

I am experiencing an issue when using the us.meta.llama3-2-90b-instruct-v1:0 model from Bedrock. The code I am using is designed to handle streaming events, specifically looking for the on_chat_model_stream event. However, when I run the code with this model, I do not receive the expected event.

I expect to receive the on_chat_model_stream event when using the us.meta.llama3-2-90b-instruct-v1:0 model, similar to how it works with the Claude 3.5 Sonnet model.

System Info

pip freeze | grep langchain
langchain==0.3.2
langchain-core==0.3.28
platform: Windows
python version: 3.11.0