Skip to content

Commit 78c2081

Browse files
z7yez7ye
committed
resolve comments
1 parent 7e86d16 commit 78c2081

File tree

1 file changed

+150
-10
lines changed

1 file changed

+150
-10
lines changed

‎docs/source/user_guide/large_language_model/retrieval.rst

Lines changed: 150 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,156 @@
11
.. _vector_store:
22

3-
########################
4-
Vector Store integration
5-
########################
3+
#################################################
4+
integration with OCI Generative AI and OpenSearch
5+
#################################################
66

77
.. versionadded:: 2.9.1
88

9-
Current version of Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a langchain application with the vector store being one of the components using data science model deployment service. To solve this problem, we extended our support of vector stores serialization:
9+
OCI Generative Embedding
10+
========================
11+
12+
The Generative AI Embedding Models convert textual input - ranging from phrases and sentences to entire paragraphs - into a structured format known as embeddings. Each piece of text input is transformed into a numerical array consisting of 1024 distinct numbers. The following pretrained model is available for creating text embeddings:
13+
14+
- embed-english-light-v2.0
15+
16+
To find out the latest supported embedding model, check the `documentation <https://docs.oracle.com/en-us/iaas/Content/generative-ai/embed-models.htm>`_.
17+
18+
The following code snippet shows how to use the Generative AI Embedding Models:
19+
20+
.. code-block:: python3
21+
22+
import ads
23+
ads.set_auth("resource_principal")
24+
25+
oci_embedings = GenerativeAIEmbeddings(
26+
compartment_id="ocid1.compartment.####",
27+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
28+
)
29+
30+
Retrieval QA with OpenSearch
31+
============================
32+
33+
OCI OpenSearch
34+
--------------
35+
36+
OCI Search with OpenSearch is a fully managed service which makes searching vast datasets and getting quick results fast and easy. In large language model world, you can use it as a vector store to store your documents and conduct keyword search or semantic search with help of a text embedding model. For a complete walk through on spinning up a OCI OpenSearch Cluster, see `Search and visualize data using OCI Search Service with OpenSearch <https://docs.oracle.com/en/learn/oci-opensearch/index.html#introduction>`_.
37+
38+
Semantic Search with OCI OpenSearch
39+
-----------------------------------
40+
41+
With the OCI OpenSearch and OCI Generative Embedding, you can do semantic search by using langchain. The following code snippet shows how to do semantic search with OCI OpenSearch:
42+
43+
.. code-block:: python3
44+
45+
import os
46+
os.environ['OCI_OPENSEARCH_USERNAME'] = "username"
47+
os.environ['OCI_OPENSEARCH_PASSWORD'] = "password"
48+
os.environ['OCI_OPENSEARCH_VERIFY_CERTS'] = "False"
49+
50+
# specify the index name that you would like to conduct semantic search on.
51+
INDEX_NAME = "your_index_name"
52+
53+
opensearch_vector_search = OpenSearchVectorSearch(
54+
"https://localhost:9200", # your oci opensearch private endpoint
55+
embedding_function=oci_embedings,
56+
index_name=INDEX_NAME,
57+
engine="lucene",
58+
http_auth=(os.environ["OCI_OPENSEARCH_USERNAME"], os.environ["OCI_OPENSEARCH_PASSWORD"]),
59+
verify_certs=os.environ["OCI_OPENSEARCH_VERIFY_CERTS"],
60+
)
61+
opensearch_vector_search.similarity_search("your query", k=2, size=2)
62+
63+
Retrieval QA Using OCI OpenSearch as a Retriever
64+
------------------------------------------------
65+
66+
Since the search result usually cannot be directly used to answer a specific question. More practical solution is to send the origiral query along with the searched results to a Large Language model to get a more coherent answer. You can also use OCI OpenSearch as a retriever for retrieval QA. The following code snippet shows how to use OCI OpenSearch as a retriever:
67+
68+
.. code-block:: python3
69+
70+
from langchain.chains import RetrievalQA
71+
from ads.llm import GenerativeAI
72+
73+
ads.set_auth("resource_principal")
74+
75+
oci_llm = GenerativeAI(
76+
compartment_id="ocid1.compartment.####",
77+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
78+
)
79+
80+
retriever = opensearch_vector_search.as_retriever(search_kwargs={"vector_field": "embeds",
81+
"text_field": "text",
82+
"k": 3,
83+
"size": 3})
84+
qa = RetrievalQA.from_chain_type(
85+
llm=oci_llm,
86+
chain_type="stuff",
87+
retriever=retriever,
88+
chain_type_kwargs={
89+
"verbose": True
90+
}
91+
)
92+
qa.run("your question")
93+
94+
FAISS as Vector DB
95+
==================
96+
97+
A lot of the time, your documents are not that large and you dont have a OCI OpenSearch cluster set up. In that case, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently.
98+
99+
The following code snippet shows how to use ``FAISS`` as your vector store:
100+
101+
.. code-block:: python3
102+
103+
from langchain.document_loaders import TextLoader
104+
from langchain.text_splitter import CharacterTextSplitter
105+
from langchain.vectorstores import FAISS
106+
107+
loader = TextLoader("your.txt")
108+
documents = loader.load()
109+
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=50)
110+
docs = text_splitter.split_documents(documents)
111+
112+
l = len(docs)
113+
embeddings = []
114+
for i in range(l // 16 + 1):
115+
subdocs = [item.page_content for item in docs[i * 16: (i + 1) * 16]]
116+
embeddings.extend(oci_embedings.embed_documents(subdocs))
117+
118+
texts = [item.page_content for item in docs]
119+
text_embedding_pairs = [(text, embed) for text, embed in zip(texts, embeddings)]
120+
db = FAISS.from_embeddings(text_embedding_pairs, oci_embedings)
121+
db.similarity_search("your query", k=2, size=2)
122+
123+
Retrieval QA Using FAISS Vector Store as a retriever
124+
----------------------------------------------------
125+
126+
Similarly, you can use FAISS Vector Store as a retriever to build a retrieval QA engine using langchain. The following code snippet shows how to use OCI OpenSearch as a retriever:
127+
128+
.. code-block:: python3
129+
130+
from langchain.chains import RetrievalQA
131+
from ads.llm import GenerativeAI
132+
133+
ads.set_auth("resource_principal")
134+
135+
oci_llm = GenerativeAI(
136+
compartment_id="ocid1.compartment.####",
137+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
138+
)
139+
retriever = db.as_retriever()
140+
qa = RetrievalQA.from_chain_type(
141+
llm=oci_llm,
142+
chain_type="stuff",
143+
retriever=retriever,
144+
chain_type_kwargs={
145+
"verbose": True
146+
}
147+
)
148+
qa.run("your question")
149+
150+
Deployment of Retrieval QA
151+
==========================
152+
153+
As of version 0.0.346, Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a Retrieval QA langchain application. To solve this problem, we extended our support of vector stores serialization:
10154

11155
- ``OpenSearchVectorSearch``
12156
- ``FAISS``
@@ -66,12 +210,12 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
66210
ads.set_auth("resource_principal")
67211
68212
oci_embedings = GenerativeAIEmbeddings(
69-
compartment_id="ocid1.compartment.oc1..aaaaaaaapvb3hearqum6wjvlcpzm5ptfxqa7xfftpth4h72xx46ygavkqteq",
213+
compartment_id="ocid1.compartment.####",
70214
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
71215
)
72216
73217
oci_llm = GenerativeAI(
74-
compartment_id="ocid1.compartment.oc1..aaaaaaaapvb3hearqum6wjvlcpzm5ptfxqa7xfftpth4h72xx46ygavkqteq",
218+
compartment_id="ocid1.compartment.####",
75219
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
76220
)
77221
@@ -123,10 +267,6 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
123267
model.predict("your prompt")
124268
125269
126-
FAISS Serialization
127-
-------------------
128-
129-
If your documents are not too large and you dont have a OCI OpenSearch cluster, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently. For ``FAISS``, you can just use it and deploy it as it is.
130270
131271
132272
FAISS Deployment

0 commit comments

Comments
 (0)