'ExLlamaV2BaseGenerator' object has no attribute 'set_stop_conditions'

[Soafp]

Description

Hi all,

I am fairly new in the langchain environment and I really need some help. What I am trying to accomplish is to make a csv chatbot using opensource LLM combined with pandas dataframe. Everything works fine except for the last step where I invoke the agent to get a response. I get 'ExLlamaV2BaseGenerator' object has no attribute 'set_stop_conditions' as an error. I tried to work around it but still no luck. Down below is the code I have been using.

Code

%%capture
!pip install exllamav2
!pip install https://github.com/turboderp/exllamav2/releases/download/v0.0.12/exllamav2-0.0.12+cu121-cp311-cp311-linux_x86_64.whl
!pip install langchain-community
!pip install langchain_experimental

from langchain.chains.llm import LLMChain
from langchain_community.llms.exllamav2 import ExLlamaV2
from huggingface_hub import snapshot_download


def download_GPTQ_model(model_name: str, models_dir: str = "./models/") -> str:
    if not os.path.exists(models_dir):
        os.makedirs(models_dir)

    _model_name = model_name.replace("/", "_")
    model_path = os.path.join(models_dir, _model_name)

    if _model_name not in os.listdir(models_dir):
        snapshot_download(repo_id=model_name, local_dir=model_path, local_dir_use_symlinks=False)
    else:
        print(f"{model_name} already exists in the models directory")

    return model_path

model_path = download_GPTQ_model("TheBloke/CapybaraHermes-2.5-Mistral-7B-GPTQ")

from exllamav2.generator import ExLlamaV2Sampler

# Set generation settings
settings = ExLlamaV2Sampler.Settings()
settings.temperature = 0.85
settings.top_k = 50
settings.top_p = 0.8
settings.token_repetition_penalty = 1.05

# Define the stop token ID (use the actual token ID for EOS or a custom stop token)
stop_token_id = 32000  # Replace this with the actual stop token ID, if needed.

# Initialize the model with stop conditions
llm = ExLlamaV2(
    model_path=model_path,  # Specify your model path
    settings=settings,
    verbose=True,
    streaming=False,
    max_new_tokens=200,
    stop_conditions=[stop_token_id]  # Pass stop token ID here
)

import pandas as pd
data = pd.read_csv("/kaggle/input/world-happiness/2019.csv")

df = pd.DataFrame(data)

from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent
#from langchain_experimental.agents import create_pandas_dataframe_agent
from langchain.agents.agent_types import AgentType
agent = create_pandas_dataframe_agent(llm, 
                                      df, 
                                      agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION, 
                                      verbose = True, 
                                      allow_dangerous_code=True)

agent.invoke("WHo is ranked first")

System Info

I am working on Kaggle with the free gpu that they provide.