Skip to content

Commit 40c3068

Browse files
z7yez7ye
authored
Update the Retrieval QA Documentation (#481)
2 parents 7e86d16 + 17c7bfa commit 40c3068

File tree

1 file changed

+164
-21
lines changed

1 file changed

+164
-21
lines changed

‎docs/source/user_guide/large_language_model/retrieval.rst

Lines changed: 164 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,167 @@
11
.. _vector_store:
22

3-
########################
4-
Vector Store integration
5-
########################
3+
#################################################
4+
Integration with OCI Generative AI and OpenSearch
5+
#################################################
66

77
.. versionadded:: 2.9.1
88

9-
Current version of Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a langchain application with the vector store being one of the components using data science model deployment service. To solve this problem, we extended our support of vector stores serialization:
9+
OCI Generative Embedding
10+
========================
11+
12+
The Generative AI Embedding Models convert textual input - ranging from phrases and sentences to entire paragraphs - into a structured format known as embeddings. Each piece of text input is transformed into a numerical array consisting of 1024 distinct numbers. The following pretrained model is available for creating text embeddings:
13+
14+
- embed-english-light-v2.0
15+
16+
To find out the latest supported embedding model, check the `documentation <https://docs.oracle.com/en-us/iaas/Content/generative-ai/embed-models.htm>`_.
17+
18+
The following code snippet shows how to use the Generative AI Embedding Models:
19+
20+
.. code-block:: python3
21+
22+
import ads
23+
ads.set_auth("resource_principal")
24+
25+
oci_embedings = GenerativeAIEmbeddings(
26+
compartment_id="ocid1.compartment.####",
27+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
28+
)
29+
30+
Retrieval QA with OpenSearch
31+
============================
32+
33+
OCI OpenSearch
34+
--------------
35+
36+
OCI Search with OpenSearch is a fully managed service which makes searching vast datasets and getting quick results fast and easy. In large language model world, you can use it as a vector store to store your documents and conduct keyword search or semantic search with help of a text embedding model. For a complete walk through on spinning up a OCI OpenSearch Cluster, see `Search and visualize data using OCI Search Service with OpenSearch <https://docs.oracle.com/en/learn/oci-opensearch/index.html#introduction>`_.
37+
38+
Semantic Search with OCI OpenSearch
39+
-----------------------------------
40+
41+
With the OCI OpenSearch and OCI Generative Embedding, you can do semantic search by using langchain. The following code snippet shows how to do semantic search with OCI OpenSearch:
42+
43+
.. code-block:: python3
44+
45+
import os
46+
os.environ['OCI_OPENSEARCH_USERNAME'] = "username"
47+
os.environ['OCI_OPENSEARCH_PASSWORD'] = "password"
48+
os.environ['OCI_OPENSEARCH_VERIFY_CERTS'] = "False"
49+
50+
# specify the index name that you would like to conduct semantic search on.
51+
INDEX_NAME = "your_index_name"
52+
53+
opensearch_vector_search = OpenSearchVectorSearch(
54+
"https://localhost:9200", # your oci opensearch private endpoint
55+
embedding_function=oci_embedings,
56+
index_name=INDEX_NAME,
57+
engine="lucene",
58+
http_auth=(os.environ["OCI_OPENSEARCH_USERNAME"], os.environ["OCI_OPENSEARCH_PASSWORD"]),
59+
verify_certs=os.environ["OCI_OPENSEARCH_VERIFY_CERTS"],
60+
)
61+
opensearch_vector_search.similarity_search("your query", k=2, size=2)
62+
63+
Retrieval QA Using OCI OpenSearch as a Retriever
64+
------------------------------------------------
65+
66+
Since the search result usually cannot be directly used to answer a specific question. More practical solution is to send the origiral query along with the searched results to a Large Language model to get a more coherent answer. You can also use OCI OpenSearch as a retriever for retrieval QA. The following code snippet shows how to use OCI OpenSearch as a retriever:
67+
68+
.. code-block:: python3
69+
70+
from langchain.chains import RetrievalQA
71+
from ads.llm import GenerativeAI
72+
73+
ads.set_auth("resource_principal")
74+
75+
oci_llm = GenerativeAI(
76+
compartment_id="ocid1.compartment.####",
77+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
78+
)
79+
80+
retriever = opensearch_vector_search.as_retriever(search_kwargs={"vector_field": "embeds",
81+
"text_field": "text",
82+
"k": 3,
83+
"size": 3})
84+
qa = RetrievalQA.from_chain_type(
85+
llm=oci_llm,
86+
chain_type="stuff",
87+
retriever=retriever,
88+
chain_type_kwargs={
89+
"verbose": True
90+
}
91+
)
92+
qa.run("your question")
93+
94+
Retrieval QA with FAISS
95+
=======================
96+
97+
FAISS as Vector DB
98+
------------------
99+
100+
A lot of the time, your documents are not that large and you dont have a OCI OpenSearch cluster set up. In that case, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently.
101+
102+
The following code snippet shows how to use ``FAISS`` along with OCI Embedding Model to do semantic search:
103+
104+
.. code-block:: python3
105+
106+
from langchain.document_loaders import TextLoader
107+
from langchain.text_splitter import CharacterTextSplitter
108+
from langchain.vectorstores import FAISS
109+
110+
loader = TextLoader("your.txt")
111+
documents = loader.load()
112+
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=50)
113+
docs = text_splitter.split_documents(documents)
114+
115+
l = len(docs)
116+
embeddings = []
117+
for i in range(l // 16 + 1):
118+
subdocs = [item.page_content for item in docs[i * 16: (i + 1) * 16]]
119+
embeddings.extend(oci_embedings.embed_documents(subdocs))
120+
121+
texts = [item.page_content for item in docs]
122+
text_embedding_pairs = [(text, embed) for text, embed in zip(texts, embeddings)]
123+
db = FAISS.from_embeddings(text_embedding_pairs, oci_embedings)
124+
db.similarity_search("your query", k=2, size=2)
125+
126+
Retrieval QA Using FAISS Vector Store as a Retriever
127+
----------------------------------------------------
128+
129+
Similarly, you can use FAISS Vector Store as a retriever to build a retrieval QA engine using langchain. The following code snippet shows how to use OCI OpenSearch as a retriever:
130+
131+
.. code-block:: python3
132+
133+
from langchain.chains import RetrievalQA
134+
from ads.llm import GenerativeAI
135+
136+
ads.set_auth("resource_principal")
137+
138+
oci_llm = GenerativeAI(
139+
compartment_id="ocid1.compartment.####",
140+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
141+
)
142+
retriever = db.as_retriever()
143+
qa = RetrievalQA.from_chain_type(
144+
llm=oci_llm,
145+
chain_type="stuff",
146+
retriever=retriever,
147+
chain_type_kwargs={
148+
"verbose": True
149+
}
150+
)
151+
qa.run("your question")
152+
153+
Deployment of Retrieval QA
154+
==========================
155+
156+
As of version 0.0.346, Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a Retrieval QA langchain application. To solve this problem, we extended our support of vector stores serialization:
10157

11158
- ``OpenSearchVectorSearch``
12159
- ``FAISS``
13160

14161
OpenSearchVectorSearch Serialization
15162
------------------------------------
16163

17-
langchain does not automatically support serialization of ``OpenSearchVectorSearch``. However, ADS provides a way to serialize ``OpenSearchVectorSearch``. To serialize ``OpenSearchVectorSearch``, you need to use environment variables to pass in the credentials. The following variables can be passed in through the corresponding environment variables:
164+
langchain does not automatically support serialization of ``OpenSearchVectorSearch``. However, ADS provides a way to serialize ``OpenSearchVectorSearch``. To serialize ``OpenSearchVectorSearch``, you need to use environment variables to store the credentials. The following variables can be passed in through the corresponding environment variables:
18165

19166
- http_auth: (``OCI_OPENSEARCH_USERNAME``, ``OCI_OPENSEARCH_PASSWORD``)
20167
- verify_certs: ``OCI_OPENSEARCH_VERIFY_CERTS``
@@ -52,10 +199,10 @@ During deployment, it is very important that you remember to pass in those envir
52199
"OCI_OPENSEARCH_PASSWORD": "<oci_opensearch_password>",
53200
"OCI_OPENSEARCH_VERIFY_CERTS": "<oci_opensearch_verify_certs>",)
54201
55-
OpenSearchVectorSearch Deployment
56-
---------------------------------
202+
Deployment of Retrieval QA with OpenSearch
203+
------------------------------------------
57204

58-
Here is an example code snippet for OpenSearchVectorSearch deployment:
205+
Here is an example code snippet for deployment of Retrieval QA using OpenSearch as a retriever:
59206

60207
.. code-block:: python3
61208
@@ -66,12 +213,12 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
66213
ads.set_auth("resource_principal")
67214
68215
oci_embedings = GenerativeAIEmbeddings(
69-
compartment_id="ocid1.compartment.oc1..aaaaaaaapvb3hearqum6wjvlcpzm5ptfxqa7xfftpth4h72xx46ygavkqteq",
216+
compartment_id="ocid1.compartment.####",
70217
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
71218
)
72219
73220
oci_llm = GenerativeAI(
74-
compartment_id="ocid1.compartment.oc1..aaaaaaaapvb3hearqum6wjvlcpzm5ptfxqa7xfftpth4h72xx46ygavkqteq",
221+
compartment_id="ocid1.compartment.####",
75222
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
76223
)
77224
@@ -107,7 +254,8 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
107254
from ads.llm.deploy import ChainDeployment
108255
model = ChainDeployment(qa)
109256
model.prepare(force_overwrite=True,
110-
inference_conda_env="your_conda_pack",
257+
inference_conda_env="<custom_conda_environment_uri>",
258+
inference_python_version="<python_version>",
111259
)
112260
113261
model.save()
@@ -123,16 +271,10 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
123271
model.predict("your prompt")
124272
125273
126-
FAISS Serialization
127-
-------------------
128-
129-
If your documents are not too large and you dont have a OCI OpenSearch cluster, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently. For ``FAISS``, you can just use it and deploy it as it is.
130-
131-
132-
FAISS Deployment
133-
----------------
274+
Deployment of Retrieval QA with FAISS
275+
-------------------------------------
134276

135-
Here is an example code snippet for FAISS deployment:
277+
Here is an example code snippet for deployment of Retrieval QA using FAISS as a retriever:
136278

137279
.. code-block:: python3
138280
@@ -182,7 +324,8 @@ Here is an example code snippet for FAISS deployment:
182324
from ads.llm.deploy import ChainDeployment
183325
model = ChainDeployment(qa)
184326
model.prepare(force_overwrite=True,
185-
inference_conda_env="your_conda_pack",
327+
inference_conda_env="<custom_conda_environment_uri>",
328+
inference_python_version="<python_version>",
186329
)
187330
188331
model.save()

0 commit comments

Comments
 (0)