Get Metadata from LLM call with structured Output

[timbmg]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import dotenv
from pydantic import BaseModel
from langchain_google_genai import ChatGoogleGenerativeAI

dotenv.load_dotenv(".env")


llm = ChatGoogleGenerativeAI(
    model="gemini-2.5-flash",
    temperature=0.6,
    max_tokens=None,
    timeout=None,
    max_retries=2,
)


class ResponseModel(BaseModel):
    response: str


prompt = "Hello, world! What is the capital of France?"


response = llm.with_structured_output(schema=ResponseModel, method="json_mode").invoke(
    prompt, include_raw=True
)

print(type(response))
# <class '__main__.ResponseModel'>

Description

I am trying to get the metadata of an LLM response. However, when using with_structured_output, it returns only the parsed response, nothing else.

System Info

➜ uv pip freeze | grep lang
google-ai-generativelanguage==0.6.18
langchain==0.3.26
langchain-core==0.3.67
langchain-google-genai==2.1.6
langchain-text-splitters==0.3.8
langgraph==0.5.1
langgraph-checkpoint==2.1.0
langgraph-prebuilt==0.5.2
langgraph-sdk==0.1.72
langsmith==0.4.4

[ccurme]

Pass include_raw=True into with_structured_output, not invoke:

response = llm.with_structured_output(
    schema=ResponseModel, method="json_mode", include_raw=True
).invoke(prompt)

print(type(response))

<class 'dict'>