Does safety settings really work in RAG langchain

[BarryC7]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

safety_settings = {
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE, 
    HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE, 
    HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE, 
    HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
}

llm = ChatGoogleGenerativeAI(model='gemini-1.5-flash',safety_settings=safety_settings,api_key=google_api_key,temperature = 0.1,max_output_tokens=3072)

# patch
ChatGoogleGenerativeAI._generate = partial(llm._generate, safety_settings=safety_settings

#normal RAG
system_prompt = (
    "You are an assistant for question-answering tasks. "
    "Use the following pieces of retrieved context to answer "
    "the question. If you don't know the answer, say that you "
    "don't know. Use three sentences maximum and keep the "
    "answer concise."
    "\n\n"
    "{context}"
)

prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("human", "{input}"),
    ]
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, question_answer_chain)

# scenario 1: SEXUALLY_EXPLICIT question with normal docs
question = "SEXUALLY_EXPLICIT_question"

normal_docs = [
    Document(page_content="""something here"""),
]

llm.invoke(question) # return properly

question_answer_chain.invoke({"input": question,'chat_history':[],"context": normal_docs})
#Gemini produced an empty response. Continuing with empty message,Feedback: block_reason: OTHER

rag_chain.invoke({"input": question,'chat_history':[''],"context": normal_docs)
#Gemini produced an empty response. Continuing with empty message,Feedback: block_reason: OTHER


#scenario 2: normal question but retriever return SEXUALLY_EXPLICIT docs

question2 = "normal question"
SEXUALLY_docs = [
    Document(page_content="""SEXUALLY_EXPLICIT"""),
]

llm.invoke(question2) # return properly

question_answer_chain.invoke({"input": question2,'chat_history':[],"context": SEXUALLY_docs})
#Gemini produced an empty response. Continuing with empty message,Feedback: block_reason: OTHER

rag_chain.invoke({"input": question2,'chat_history':[''],"context": SEXUALLY_docs)
#Gemini produced an empty response. Continuing with empty message,Feedback: block_reason: OTHER

Description

I tried to build a RAG with langchain for question answering, I read the following references：
#15095
https://python.langchain.com/docs/tutorials/rag/

but I found that it still returns an empty result‘ ’ with safety settings.

There are two situations, 1st is SEXUALLY_EXPLICIT question with normal docs, 2nd is normal question but retriever return SEXUALLY_EXPLICIT docs. both will return empty results with block_reason: OTHER.

I want to know if this is a problem with Google, a problem with langchain, or a problem with myself.

System Info

langchain 0.3.1
langchain-community 0.3.1
langchain-core 0.3.6
langchain-experimental 0.3.2
langchain-google-genai 2.0.0
google-ai-generativelanguage 0.6.5
google-generativeai 0.7.0

[dosubot[bot]]

The issue of receiving empty responses with a block reason of "OTHER" in LangChain's RAG setup when using the ChatGoogleGenerativeAI model with safety settings likely stems from the safety settings configuration. The Gemini models have default safety settings that can be overridden. If you are receiving lots of "Safety Warnings" from your models, you can try tweaking the safety_settings attribute of the model.

Here is an example of how to adjust the safety settings:

from langchain_google_genai import (
    ChatGoogleGenerativeAI,
    HarmBlockThreshold,
    HarmCategory,
)

llm = ChatGoogleGenerativeAI(
    model="gemini-1.5-flash",
    google_api_key=google_api_key,
    safety_settings={
        HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
        HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
        HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
        HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
    },
)

For an enumeration of the categories and thresholds available, see Google's safety setting types ^[1].

Additionally, the LangChain library includes a retry mechanism for handling API exceptions, which might be useful if the issue is related to transient errors from the Google PaLM API. The retry mechanism is implemented using the tenacity library and can handle exceptions like ResourceExhausted, ServiceUnavailable, and GoogleAPIError.

Here is an example of how to use the retry mechanism:

from langchain_community.chat_models import ChatGooglePalm

def chat_with_retry(llm: ChatGooglePalm, **kwargs: Any) -> Any:
    """Use tenacity to retry the completion call."""
    retry_decorator = _create_retry_decorator()

    @retry_decorator
    def _chat_with_retry(**kwargs: Any) -> Any:
        return llm.client.chat(**kwargs)

    return _chat_with_retry(**kwargs)

Therefore, the issue is likely related to the safety settings configuration in your implementation, but it could also be influenced by transient API errors that can be mitigated using the retry mechanism provided by LangChain ^[2][3].

To continue talking to Dosu, mention @dosu.

[jvlinsta]

https://www.reddit.com/r/SillyTavernAI/comments/1jorzqg/gemini_25_pro_experimental_giving_you_headache/
This is relevant ;)
Just crank up max_tokens