Getting None respone from ChatOllama in tagging exercise

[uzairlatif90]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

class Classification(BaseModel):
    Sentiment: Optional[str] = Field(
        default="No", 
        description="Describes the emotion of the customers",
        enum=["Positive", "Neutral", "Negative"])
    Language: Optional[str] = Field(
        default="No", enum=["french", "dutch"]
    )
    Customer_Contract: Optional[str] = Field(#'CUSTOMER - Contract'
        default="No",
        description="""A flag to check if the customer mentions their contract, wants a new contract, wants to terminate the contract, expresses interest in a specific type of contract or discusses the end of their current contract and express interest in a renewal.""",
        enum=["Yes","No"],
    )

    Follow_up: Optional[str] = Field(#'Follow-up'
        default="No",
        description="""A flag to check if the customer is following up on a previous request or complaint made by them.""",
        enum=["Yes","No"],
    )
    FAQLink: Optional[str] = Field(#'FAQ Link'
        default="No",
        description="""A flag to check every time the agent uses copy and pastes the link to the FAQ on the the website or uses a sentence or explanation from the FAQ section on the website.""",
        enum=["Yes","No"],
    )

#initialize and set the ollama model
from langchain_ollama import ChatOllama
llm = ChatOllama(model="mistral-nemo",temperature=0,num_ctx=4096,verbose=True).with_structured_output(
    Classification
)
modelName="mistralnemo"

tagging_prompt = ChatPromptTemplate.from_template(
"""
You are an expert extraction assistant for extracting insights from conversations between a customer and an agent.

You will be given the transcript of a conversation between a customer and an agent.

Extract the desired information from the following conversation.

Only extract the properties and check for the flags mentioned in the 'Classification' function.

Conversation between agent and customer:
{input}
"""
)

tagging_chain = tagging_prompt | llm

response=tagging_chain.invoke({"input": conversationContent})

Description

I am trying to use Langchain with ChatOllama to tag customer conversation wtih different tags based on a Classification schema. When I invoke the chain, sometimes I get a proper response from the LLM but somtimes I get a None response from the chain without any explanation of what might have caused it. How can I know what is causing a None respone and how can I correct for that?

System Info

System Information

OS: Linux
OS Version: #202405180028 SMP PREEMPT_DYNAMIC Sat May 18 00:46:12 UTC 2024
Python Version: 3.10.13 (main, Aug 19 2024, 22:31:41) [GCC 11.4.0]

Package Information

langchain_core: 0.2.35
langchain: 0.2.14
langchain_community: 0.2.12
langsmith: 0.1.104
langchain_experimental: 0.0.64
langchain_ollama: 0.1.1
langchain_text_splitters: 0.2.2

Optional packages not installed

langgraph
langserve

Other Dependencies

aiohttp: 3.10.5
async-timeout: 4.0.3
dataclasses-json: 0.6.7
httpx: 0.27.0
jsonpatch: 1.33
numpy: 1.26.4
ollama: 0.3.1
orjson: 3.10.7
packaging: 24.1
pydantic: 2.8.2
PyYAML: 6.0.2
requests: 2.32.3
SQLAlchemy: 2.0.32
tenacity: 8.5.0
typing-extensions: 4.12.2

[dosubot[bot]]

Hey there, @uzairlatif90! I'm here to help you with any issues you're facing. I'm a bot designed to assist you with bug fixes, answer your questions, and guide you on becoming a contributor. Let's tackle this challenge together! 🤖

To determine the cause of a None response from the LangChain ChatOllama tagging chain and correct it, you should follow these steps:

1. Check the JSON Response: Ensure that the response from the model is valid JSON. The _generate and _agenerate methods attempt to parse the response as JSON. If the response is not valid JSON, it will raise a ValueError.

2. Verify Tool Calls: Ensure that the tool specified in the response exists and is correctly defined. The code checks for the tool name in the parsed JSON response and matches it with the provided tools. If the tool is not found or is incorrectly defined, it will raise a ValueError.

3. Check for Default Response: If no specific tool is called, the default response function (__conversational_response) should be used. Ensure that this default response is correctly formatted and included in the functions list.

Here is a code snippet that highlights the key areas to check:

def _generate(
        self,
        messages: List[BaseMessage],
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> ChatResult:
    functions = kwargs.get("functions", [])
    if "functions" in kwargs:
        del kwargs["functions"]
    if "function_call" in kwargs:
        functions = [
            fn for fn in functions if fn["name"] == kwargs["function_call"]["name"]
        ]
        if not functions:
            raise ValueError(
                "If `function_call` is specified, you must also pass a "
                "matching function in `functions`."
            )
        del kwargs["function_call"]
    functions = [convert_to_ollama_tool(fn) for fn in functions]
    functions.append(DEFAULT_RESPONSE_FUNCTION)
    system_message_prompt_template = SystemMessagePromptTemplate.from_template(
        self.tool_system_prompt_template
    )
    system_message = system_message_prompt_template.format(
        tools=json.dumps(functions, indent=2)
    )
    response_message = super()._generate(
        [system_message] + messages, stop=stop, run_manager=run_manager, **kwargs
    )
    chat_generation_content = response_message.generations[0].text
    if not isinstance(chat_generation_content, str):
        raise ValueError("OllamaFunctions does not support non-string output.")
    try:
        parsed_chat_result = json.loads(chat_generation_content)
    except json.JSONDecodeError:
        raise ValueError(
            f"""'{self.model}' did not respond with valid JSON. 
            Please try again. 
            Response: {chat_generation_content}"""
        )
    called_tool_name = (
        parsed_chat_result["tool"] if "tool" in parsed_chat_result else None
    )
    called_tool = next(
        (fn for fn in functions if fn["name"] == called_tool_name), None
    )
    if (
        called_tool is None
        or called_tool["name"] == DEFAULT_RESPONSE_FUNCTION["name"]
    ):
        if (
            "tool_input" in parsed_chat_result
            and "response" in parsed_chat_result["tool_input"]
        ):
            response = parsed_chat_result["tool_input"]["response"]
        elif "response" in parsed_chat_result:
            response = parsed_chat_result["response"]
        else:
            raise ValueError(
                f"Failed to parse a response from {self.model} output: "
                f"{chat_generation_content}"
            )
        return ChatResult(
            generations=[
                ChatGeneration(
                    message=AIMessage(
                        content=response,
                    )
                )
            ]
        )

By following these steps and checking the relevant parts of the code, you should be able to identify and correct the cause of a None response ^[1].

To continue talking to Dosu, mention @dosu.