How to replace gpt-3.5-turbo-instruct with other OpenAI models for ContextualCompressionRetriever?

[bing0037]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import LLMChainExtractor
from langchain_openai import OpenAI

llm = OpenAI(model='gpt-4', temperature=0) # default model is 'gpt-3.5-turbo-instruct'!
compressor = LLMChainExtractor.from_llm(llm)
compression_retriever = ContextualCompressionRetriever(
    base_compressor=compressor, base_retriever=retriever
)

compressed_docs = compression_retriever.invoke(
    "What did the president say about Ketanji Jackson Brown"
)
print(compressed_docs)

Description

The default OpenAI LLM (llm = OpenAI(temperature=0) is 'gpt-3.5-turbo-instruct', but I can't access 'gpt-3.5-turbo-instruct' for some reason and used 'gpt-4' instead. This leads to error.
Is there any other models that can be used for ContextualCompressionRetriever?

Here is the error:

openai.BadRequestError: Error code: 400 - {'error': {'message': 'The completion operation does not work with the specified model, gpt-4. Please choose different model and try again.

System Info

System Information

OS: Windows
OS Version: 10.0.22631
Python Version: 3.12.5 | packaged by conda-forge | (main, Aug 8 2024, 18:24:51) [MSC v.1940 64 bit (AMD64)]

Package Information

langchain_core: 0.2.38
langchain: 0.2.16
langchain_community: 0.2.16
langsmith: 0.1.106
langchain_chroma: 0.1.3
langchain_experimental: 0.0.65
langchain_openai: 0.1.22
langchain_text_splitters: 0.2.2

Optional packages not installed

langgraph
langserve

Other Dependencies

aiohttp: 3.10.5
async-timeout: 4.0.3
chromadb: 0.5.3
dataclasses-json: 0.6.7
fastapi: 0.113.0
httpx: 0.27.2
jsonpatch: 1.33
numpy: 1.26.4
openai: 1.42.0
orjson: 3.10.7
packaging: 24.1
pydantic: 2.8.2
PyYAML: 6.0.2
requests: 2.32.3
SQLAlchemy: 2.0.32
tenacity: 8.5.0
tiktoken: 0.7.0
typing-extensions: 4.12.2

[dosubot[bot]]

Hey there, @bing0037!
I'm here to help you out with any bugs, questions, or contributions you might have. Let's tackle this together!

I found a similar solved discussion that might be helpful: Issue with LangChain Misclassifying gpt-3.5-turbo-instruct as Chat Model. The solution suggested there is to upgrade to a newer version of LangChain, as the model handling logic has been updated in later versions ^[1].

Regarding your specific question about using other models with ContextualCompressionRetriever, you can indeed use other models. Here are some examples:

1. Using RankZephyr:

   from langchain.retrievers.contextual_compression import ContextualCompressionRetriever
   from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank
   
   compressor = RankLLMRerank(top_n=3, model="zephyr")
   compression_retriever = ContextualCompressionRetriever(
       base_compressor=compressor, base_retriever=retriever
   )
   
   compressed_docs = compression_retriever.invoke("What was done to Russia?")
   pretty_print_docs(compressed_docs)

2. Using RankGPT:

   from langchain.retrievers.contextual_compression import ContextualCompressionRetriever
   from langchain_community.document_compressors.rankllm_rerank import RankLLMRerank
   
   compressor = RankLLMRerank(top_n=3, model="gpt", gpt_model="gpt-3.5-turbo")
   compression_retriever = ContextualCompressionRetriever(
       base_compressor=compressor, base_retriever=retriever
   )
   
   compressed_docs = compression_retriever.invoke("What did the president say about Ketanji Brown Jackson")
   pretty_print_docs(compressed_docs)

These configurations allow you to use different models effectively with ContextualCompressionRetriever ^[2][3].

To continue talking to Dosu, mention @dosu.

[bing0037]

Actually, this is a different problem with #15598 (comment).. The problem you menthioned is how to use gpt-3.5-turbo-instruct, while my problem is how to replace gpt-3.5-turbo-instruct with other models.

Also, my current langchain version is 0.2.16, greater than langchain 0.0.130 mentioned in #15598 (comment).

Lastly, for RankZephyr and RankGPT, I failed while installing rank_llm using pip install rank_llm in Windows Powershell and I will try on Linux later.

[mspronesti]

The problem is that OpenAI only supports completion models, while you are using a chat completion model (gpt-4). Try this:

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model='gpt-4', temperature=0)