docs/integrations/document_transformers/cross_encoder_reranker/

[giscus[bot]]

docs/integrations/document_transformers/cross_encoder_reranker/

This notebook shows how to implement reranker in a retriever with your own cross encoder from Hugging Face cross encoder models or Hugging Face models that implements cross encoder function (example: BAAI/bge-reranker-base). SagemakerEndpointCrossEncoder enables you to use these HuggingFace models loaded on Sagemaker.

https://python.langchain.com/docs/integrations/document_transformers/cross_encoder_reranker/

[pedrorjbr]

Instead of using a local model from HuggingFace, I would like to use the infered endpoints. How can I do that?

    index = self.pc.Index(self.index_name)

    embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
    vector_store = PineconeVectorStore(index=index, embedding=embeddings)

    dense_retriever = vector_store.as_retriever(
        search_type="similarity", search_kwargs={"k": 5}
    )
    cross_encoder_model = HuggingFaceEndpoint(
        endpoint_url="https://ugwu6jjlu82z8gxm.us-east-1.aws.endpoints.huggingface.cloud",
        max_new_tokens=512,
        top_k=10,
        top_p=0.95,
        typical_p=0.95,
        temperature=0.01,
        repetition_penalty=1.03,
        huggingfacehub_api_token="XXX"
        
    )
    
    
    # cross_encoder_model = HuggingFaceCrossEncoder(
    #     model_name="BAAI/bge-reranker-base"
    # )
    reranker = CrossEncoderReranker(model=cross_encoder_model, top_n=3)

    retriever = ContextualCompressionRetriever(
        base_compressor=reranker, base_retriever=dense_retriever
    )