Structured Output Fails with ChatTogether Model

[mail2mhossain]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_together import ChatTogether
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from decouple import config

TOGETHER_API_KEY = config("TOGETHER_API_KEY")

llm = ChatTogether(
    together_api_key=TOGETHER_API_KEY,
    model= "meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo",
    temperature=0,
)


class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

chain = llm.with_structured_output(Joke) 


results = chain.invoke("Tell me a joke about horse.")
print(results)

Description

I am trying to get a structured output from ChatTogether but failed. Getting following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': 'meta-llama/Llama-3.2-90B-Vision-Instruct-Turbo is not supported for JSON mode/function calling', 'type': 'invalid_request_error', 'param': None, 'code': 'constraints_model'}}

System Info

langchain Version: 0.3.2
langchain-together Version: 0.2.0
openai Version: 1.45.1

Windows 11

Python Version: 3.11.9

[viotemp1]

I have the same issue with other models.

• meta-llama/Llama-3-70b-chat-hf
• deepseek-ai/DeepSeek-R1
• deepseek-ai/DeepSeek-V3

How to solve?

[ccurme]

Structured output by default uses function calling under the hood. You need to use a model that supports this feature. Together lists them here. For example, "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo" works with your example.