Exception: AttributeError: 'int' object has no attribute 'name'

[rvasa779]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import os

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from langchain.prompts import PromptTemplate
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.runnables import ConfigurableField
from langchain_google_genai import (ChatGoogleGenerativeAI, HarmBlockThreshold,
                                    HarmCategory)
from langchain_openai import AzureChatOpenAI

# Setting up APIs
# Azure OpenAI API
OPENAI_API_KEY = os.environ["OPENAI_API_KEY"]
OPENAI_DEPLOYMENT_ENDPOINT = os.environ["AZURE_OPENAI_ENDPOINT"]
OPENAI_DEPLOYMENT_NAME = os.environ["OPENAI_DEPLOYMENT_NAME"]
OPENAI_DEPLOYMENT_VERSION = os.environ["OPENAI_API_VERSION"]


# Models
llm = AzureChatOpenAI(azure_endpoint=OPENAI_DEPLOYMENT_ENDPOINT,
        deployment_name=OPENAI_DEPLOYMENT_NAME,
        openai_api_version=OPENAI_DEPLOYMENT_VERSION,
        openai_api_key=OPENAI_API_KEY, verbose=True, 
        request_timeout=60, temperature=0.8).configurable_alternatives(
    # This gives this field an id
    # When configuring the end runnable, we can then use this id to configure this field
    ConfigurableField(id="llm"),
    # This sets a default_key.
    # If we specify this key, the default LLM (ChatAnthropic initialized above) will be used
    default_key="azure",
    # This adds a new option, with name `openai` that is equal to `ChatOpenAI()`
    gemini=ChatGoogleGenerativeAI(
        model="gemini-pro",
        google_api_key="<API_KEY>",
        convert_system_message_to_human=True,
        max_tokens=16384,
        temperature=0.7,
        safety_settings={
            HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
            HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
            HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE
        },
        ),
)



app = FastAPI(title="Joke Engine", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class Joke(BaseModel):
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

##Generic home end point
@app.get("/")
def home():
    return {"msg": "App is running successfully"}

@app.get("/joke")
async def run_agent(topic: str):
    
    prompt = PromptTemplate(
                input_variables=["input",],
                template="""
                Given a user input, tell a joke about the below topic:
                
                {input}
                
                """
                )
    
    chain = prompt | llm
    return chain.with_config(configurable={"llm": "gemini"}).invoke({'input':topic})

Description

I am using ChatGoogleGenerativeAI along with AzureOpenAI in configurable mode.

I have written a simple fastapi to write a joke on user given topic. When I run the above code configuring my "llm" parameter as azure it works fine. However, when I run the same code with "llm" parameter as gemini the code fails with the below error:

[2024-06-14T05:30:47.913Z] System.Private.CoreLib: Exception while executing function: Functions.HttpTrigger1. System.Private.CoreLib: Result: Failure Exception: AttributeError: 'int' object has no attribute 'name' Stack: File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 479, in _handle__invocation_request call_result = await self._loop.run_in_executor( File "C:\Users\Digital\AppData\Local\Programs\Python\Python39\lib\concurrent\futures\thread.py", line 52, in run result = self.fn(*self.args, **self.kwargs) File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure_functions_worker\dispatcher.py", line 752, in _run_sync_func return ExtensionManager.get_sync_invocation_wrapper(context, File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure_functions_worker\extension.py", line 215, in _raw_invocation_wrapper result = function(**args) File "<code_path>\egg-gemini\HttpTrigger1\__init__.py", line 19, in main return func.AsgiMiddleware(app).handle(req, context) File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure\functions\_http_asgi.py", line 172, in handle return self._handle(req, context) File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure\functions\_http_asgi.py", line 177, in _handle asgi_response = asyncio.run( File "C:\Users\Digital\AppData\Local\Programs\Python\Python39\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Users\Digital\AppData\Local\Programs\Python\Python39\lib\asyncio\base_events.py", line 642, in run_until_complete return future.result() File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9\WINDOWS\X64\azure\functions\_http_asgi.py", line 80, in from_app await app(scope, res._receive, res._send) File "<code_path>\egg-gemini\.venv\lib\site-packages\fastapi\applications.py", line 1054, in __call__ await super().__call__(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\applications.py", line 123, in __call__ await self.middleware_stack(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\middleware\errors.py", line 186, in __call__ raise exc File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\middleware\errors.py", line 164, in __call__ await self.app(scope, receive, _send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\middleware\cors.py", line 85, in __call__ await self.app(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\middleware\exceptions.py", line 65, in __call__ await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app raise exc File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\routing.py", line 756, in __call__ await self.middleware_stack(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\routing.py", line 776, in app await route.handle(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\routing.py", line 297, in handle await self.app(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app raise exc File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "<code_path>\egg-gemini\.venv\lib\site-packages\starlette\routing.py", line 72, in app response = await func(request) File "<code_path>\egg-gemini\.venv\lib\site-packages\fastapi\routing.py", line 278, in app raw_response = await run_endpoint_function( File "<code_path>\egg-gemini\.venv\lib\site-packages\fastapi\routing.py", line 191, in run_endpoint_function return await dependant.call(**values) File "<code_path>\egg-gemini\src\__init__.py", line 100, in run_agent return chain.with_config(configurable={"llm": "gemini"}).invoke({'input':topic}) File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\runnables\base.py", line 4573, in invoke return self.bound.invoke( File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\runnables\base.py", line 2504, in invoke input = step.invoke(input, config) File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\runnables\configurable.py", line 117, in invoke return runnable.invoke(input, config, **kwargs) File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\language_models\chat_models.py", line 170, in invoke self.generate_prompt( File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\language_models\chat_models.py", line 599, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs) File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\language_models\chat_models.py", line 456, in generate raise e File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\language_models\chat_models.py", line 446, in generate self._generate_with_cache( File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_core\language_models\chat_models.py", line 671, in _generate_with_cache result = self._generate( File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_google_genai\chat_models.py", line 766, in _generate return _response_to_result(response) File "<code_path>\egg-gemini\.venv\lib\site-packages\langchain_google_genai\chat_models.py", line 551, in _response_to_result generation_info["finish_reason"] = candidate.finish_reason.name

Attached is my requirement.txt file too
requirements.txt

System Info

System Information

OS: Windows
OS Version: 10.0.19041
Python Version: 3.9.0 (tags/v3.9.0:9cf6752, Oct 5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]

Package Information

langsmith: 0.1.77
langchain_google_genai: 1.0.6
langchain_openai: 0.1.8
langchain_text_splitters: 0.2.1

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langgraph
langserve

[dosubot[bot]]

Hello, @rvasa779! I'm here to assist you. If you need help with bugs, questions, or becoming a contributor, I'm the bot for the job.

The AttributeError: 'int' object has no attribute 'name' error you're encountering when using the gemini model is likely due to an issue with the structure or type of the data being passed to the ChatGoogleGenerativeAI model. Specifically, the finish_reason attribute in the _response_to_result function is expected to have a name attribute, but it seems to be an integer instead.

To resolve this issue, you need to ensure that the finish_reason attribute is correctly set and has the expected structure. Here is a potential fix:

1. Check the finish_reason Attribute: Ensure that the finish_reason attribute in the response from the gemini model is not an integer. It should be an object with a name attribute.

2. Modify the _response_to_result Function: Update the _response_to_result function to handle cases where finish_reason might be an integer. You can add a check to convert it to a string or handle it appropriately.

Here is an example of how you might modify the _response_to_result function:

def _response_to_result(response):
    result = {}
    for candidate in response.candidates:
        generation_info = {}
        if hasattr(candidate.finish_reason, 'name'):
            generation_info["finish_reason"] = candidate.finish_reason.name
        else:
            generation_info["finish_reason"] = str(candidate.finish_reason)
        result["generation_info"] = generation_info
    return result

This modification ensures that if finish_reason is an integer, it will be converted to a string, avoiding the AttributeError.

Additionally, ensure that the data_dict being passed to the img_prompt_func function contains the correct structure and types. Specifically, the data_dict["context"]["images"] should be a list of base64-encoded image strings, and data_dict['question'] should be a string.

Here is the relevant part of the code that needs to be checked and possibly corrected:

def img_prompt_func(data_dict, num_images=2):
    messages = []
    if data_dict["context"]["images"]:
        for image in data_dict["context"]["images"][:num_images]:
            image_message = {
                "type": "image_url",
                "image_url": {"url": f"data:image/jpeg;base64,{image}"},
            }
            messages.append(image_message)
    text_message = {
        "type": "text",
        "text": (
            "You are an analyst tasked with answering questions about visual content.\n"
            "You will be given a set of image(s) from a slide deck / presentation.\n"
            "Use this information to answer the user question. \n"
            f"User-provided question: {data_dict['question']}\n\n"
        ),
    }
    messages.append(text_message)
    return [HumanMessage(content=messages)]

Ensure that the data_dict passed to this function is structured correctly. For example:

data_dict = {
    "context": {
        "images": ["base64_encoded_image_string1", "base64_encoded_image_string2"]
    },
    "question": "What is shown in these images?"
}

If data_dict["context"]["images"] contains integers or any other type that does not have a name attribute, it will cause the AttributeError. Make sure that the images are properly base64-encoded strings ^[1].

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

The same code generates different types of response objects when invoked via Azure Function versus FastAPI because the response_content_type attribute in the GenericRequestsWrapper class determines the type of response content returned. This attribute can be set to either "text" or "json".

To ensure consistent behavior across both invocation methods, you need to explicitly set the response_content_type to the desired type ("text" or "json") in your wrapper class. For example, if you want to always return JSON responses, you should use the JsonRequestsWrapper class, which sets response_content_type to "json".

Here is an example of how to use the JsonRequestsWrapper to ensure consistent JSON responses:

from your_module import JsonRequestsWrapper

# Create an instance of JsonRequestsWrapper
requests_wrapper = JsonRequestsWrapper()

# Use the wrapper to make a GET request
response = requests_wrapper.get("https://api.example.com/data")

# The response will always be in JSON format
print(response)

Similarly, if you want to always return text responses, you should use the TextRequestsWrapper class:

from your_module import TextRequestsWrapper

# Create an instance of TextRequestsWrapper
requests_wrapper = TextRequestsWrapper()

# Use the wrapper to make a GET request
response = requests_wrapper.get("https://api.example.com/data")

# The response will always be in text format
print(response)

By using the appropriate wrapper class, you can ensure that the response type is consistent regardless of whether the code is invoked via Azure Function or FastAPI ^[1].

To continue talking to Dosu, mention @dosu.

[rvasa779]

@eyurtsev requesting your help on this. I am unable to figure of the working of this.

[rvasa779]

Hi @eyurtsev , @dosu,

I have downgraded few dependency for it to work:

googleapis-common-protos==1.56.2
protobuf==3.20.3

The latest versions are not working.

[dosubot[bot]]

A maintainer has already been looped in.

To continue talking to Dosu, mention @dosu.

[StavC]

Hi @eyurtsev , @dosu,

I have downgraded few dependency for it to work:

googleapis-common-protos==1.56.2 protobuf==3.20.3

The latest versions are not working.

Thanks, worked out for me aswell. as I encountered the same problem today.

[baskaryan]

transferring to langchain-google repo for visibility, cc @lkuligin