Using OpenRouter LLM Class to generate structured output

[xxchen04120412]

Hi,

I've created a custom LLM class that inherits the properties of an OpenAI class (This is from multiple community posts)

# Inherits from ChatOpenAI since OpenRouter uses OpenAI-compatible API
class ChatOpenRouter(ChatOpenAI):
    # Define how to handle the API key
    openai_api_key: Optional[SecretStr] = Field(
        alias="api_key",  # allows using 'api_key' as parameter name
        default_factory=secret_from_env("OPENROUTER_API_KEY", default=None)  # tries to get key from environment
    )
    
    # Define which environment variable contains the secret
    @property
    def lc_secrets(self) -> dict[str, str]:
        return {"openai_api_key": "OPENROUTER_API_KEY"}

    # Constructor for the class
    def __init__(self,
                 openai_api_key: Optional[str] = None,
                 **kwargs):
        # Get API key from parameter or environment
        openai_api_key = openai_api_key or os.environ.get("OPENROUTER_API_KEY")
        
        # Initialize parent class with OpenRouter's base URL
        super().__init__(
            #base_url="https://openrouter.ai/api/v1",  # OpenRouter's API endpoint
            base_url = os.environ.get("OPENROUTER_BASE_URL"),
            openai_api_key=openai_api_key,  # Pass the API key
            **kwargs  # Pass any other parameters
        )

However, when I try to use pydantic BaseModel to call my OpenRouter API to generate a sturctured output, it FAILS

The test code is trying to "tag" a user input and classify into one of the available labels.

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field

tagging_prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a financial analyst. Extract the investment topic from the comment provided."),
    ("user", "{input}")
])

class Classification(BaseModel):
    sentiment: str = Field(description="The sentiment of the text")
    # aggressiveness: int = Field(
    #     description="How aggressive the text is on a scale from 1 to 10"
    # )
    topic: str = Field(
        ...,
        description="The investment topic or theme that the comment discusses",
        enum = ["EWZ / Brazil ETF",
          "US Treasury Yield",
          "Europe Equities",
          "MAG7 / NASDAQ",
          "China Equities"
          ])

# set up ChatOpenRouter class
#model_name = "deepseek/deepseek-r1-zero:free"
#model_name = "google/gemini-2.0-pro-exp-02-05:free"
model_name = "anthropic/claude-3.7-sonnet:thinking"

openrouter_model = ChatOpenRouter(
    model_name=model_name,
    temperature=0,
    streaming=False
)


structured_llm = openrouter_model.with_structured_output(
    Classification
)

inp = {'created_at': '2025-03-07T17:07:10.189Z', 'user_id': 29266138, 'body': 'You did this last two weeks? Or putting on now?\n\nAlso biasing to Dax and China and Brazil. Need to raise cash tho :/', 
       'user_name': 'xx', 'user_prefix': ''}
prompt = tagging_prompt.invoke({"input": inp['body']})
result = structured_llm.invoke(prompt)
print(result)

By looking at the traceback, it seems that Classification is receiving an invalid json response. even though the model selected EXPLICITLY allows for structured input.

 Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value='The investment topic fro... liquidity constraints.', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/json_invalid