You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/user_guide/large_language_model/deploy_langchain_application.rst
+8-5Lines changed: 8 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -68,15 +68,18 @@ In this example, we're using a temporary folder generated by ``tempfile.mkdtemp(
68
68
Prepare the Model Artifacts
69
69
***************************
70
70
71
-
Call ``prepare()`` from ``ChainDeployment`` to generate the ``score.py`` and serialize the LangChain application to ``chain.yaml`` file under ``artifact_dir`` folder.
72
-
Parameters ``inference_conda_env`` and ``inference_python_version`` are passed to define the conda environment where your LangChain application will be running on OCI.
73
-
Here we're using ``pytorch21_p39_gpu_v1`` with python 3.9.
71
+
Call ``prepare`` from ``ChainDeployment`` to generate the ``score.py`` and serialize the LangChain application to ``chain.yaml`` file under ``artifact_dir`` folder.
72
+
Parameters ``inference_conda_env`` and ``inference_python_version`` are passed to define the conda environment where your LangChain application will be running on OCI cloud.
73
+
Here, replace ``custom_conda_environment_uri`` with your conda environment uri that has the latest ADS 2.9.1 and replace ``python_version`` with your conda environment python version.
74
+
75
+
.. note::
76
+
For how to customize and publish conda environment, take reference to `Publishing a Conda Environment to an Object Storage Bucket <https://docs.oracle.com/en-us/iaas/data-science/using/conda_publishs_object.htm>`_
Current version of Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a langchain application with the vector store being one of the components using data science model deployment service. To solve this problem, we extended our support of vector stores serialization:
9
+
OCI Generative Embedding
10
+
========================
11
+
12
+
The Generative AI Embedding Models convert textual input - ranging from phrases and sentences to entire paragraphs - into a structured format known as embeddings. Each piece of text input is transformed into a numerical array consisting of 1024 distinct numbers. The following pretrained model is available for creating text embeddings:
13
+
14
+
- embed-english-light-v2.0
15
+
16
+
To find out the latest supported embedding model, check the `documentation <https://docs.oracle.com/en-us/iaas/Content/generative-ai/embed-models.htm>`_.
17
+
18
+
The following code snippet shows how to use the Generative AI Embedding Models:
19
+
20
+
.. code-block:: python3
21
+
22
+
import ads
23
+
ads.set_auth("resource_principal")
24
+
25
+
oci_embedings = GenerativeAIEmbeddings(
26
+
compartment_id="ocid1.compartment.####",
27
+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
28
+
)
29
+
30
+
Retrieval QA with OpenSearch
31
+
============================
32
+
33
+
OCI OpenSearch
34
+
--------------
35
+
36
+
OCI Search with OpenSearch is a fully managed service which makes searching vast datasets and getting quick results fast and easy. In large language model world, you can use it as a vector store to store your documents and conduct keyword search or semantic search with help of a text embedding model. For a complete walk through on spinning up a OCI OpenSearch Cluster, see `Search and visualize data using OCI Search Service with OpenSearch <https://docs.oracle.com/en/learn/oci-opensearch/index.html#introduction>`_.
37
+
38
+
Semantic Search with OCI OpenSearch
39
+
-----------------------------------
40
+
41
+
With the OCI OpenSearch and OCI Generative Embedding, you can do semantic search by using langchain. The following code snippet shows how to do semantic search with OCI OpenSearch:
Since the search result usually cannot be directly used to answer a specific question. More practical solution is to send the origiral query along with the searched results to a Large Language model to get a more coherent answer. You can also use OCI OpenSearch as a retriever for retrieval QA. The following code snippet shows how to use OCI OpenSearch as a retriever:
67
+
68
+
.. code-block:: python3
69
+
70
+
from langchain.chains import RetrievalQA
71
+
from ads.llm import GenerativeAI
72
+
73
+
ads.set_auth("resource_principal")
74
+
75
+
oci_llm = GenerativeAI(
76
+
compartment_id="ocid1.compartment.####",
77
+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
A lot of the time, your documents are not that large and you dont have a OCI OpenSearch cluster set up. In that case, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently.
101
+
102
+
The following code snippet shows how to use ``FAISS`` along with OCI Embedding Model to do semantic search:
103
+
104
+
.. code-block:: python3
105
+
106
+
from langchain.document_loaders import TextLoader
107
+
from langchain.text_splitter import CharacterTextSplitter
Similarly, you can use FAISS Vector Store as a retriever to build a retrieval QA engine using langchain. The following code snippet shows how to use OCI OpenSearch as a retriever:
130
+
131
+
.. code-block:: python3
132
+
133
+
from langchain.chains import RetrievalQA
134
+
from ads.llm import GenerativeAI
135
+
136
+
ads.set_auth("resource_principal")
137
+
138
+
oci_llm = GenerativeAI(
139
+
compartment_id="ocid1.compartment.####",
140
+
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
141
+
)
142
+
retriever = db.as_retriever()
143
+
qa = RetrievalQA.from_chain_type(
144
+
llm=oci_llm,
145
+
chain_type="stuff",
146
+
retriever=retriever,
147
+
chain_type_kwargs={
148
+
"verbose": True
149
+
}
150
+
)
151
+
qa.run("your question")
152
+
153
+
Deployment of Retrieval QA
154
+
==========================
155
+
156
+
As of version 0.0.346, Langchain does not support serialization of any vector stores. This will be a problem when you want to deploy a Retrieval QA langchain application. To solve this problem, we extended our support of vector stores serialization:
10
157
11
158
- ``OpenSearchVectorSearch``
12
159
- ``FAISS``
13
160
14
161
OpenSearchVectorSearch Serialization
15
162
------------------------------------
16
163
17
-
langchain does not automatically support serialization of ``OpenSearchVectorSearch``. However, ADS provides a way to serialize ``OpenSearchVectorSearch``. To serialize ``OpenSearchVectorSearch``, you need to use environment variables to pass in the credentials. The following variables can be passed in through the corresponding environment variables:
164
+
langchain does not automatically support serialization of ``OpenSearchVectorSearch``. However, ADS provides a way to serialize ``OpenSearchVectorSearch``. To serialize ``OpenSearchVectorSearch``, you need to use environment variables to store the credentials. The following variables can be passed in through the corresponding environment variables:
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
client_kwargs=dict(service_endpoint="https://generativeai.aiservice.us-chicago-1.oci.oraclecloud.com") # this can be omitted after Generative AI service is GA.
76
223
)
77
224
@@ -95,8 +242,7 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
@@ -124,16 +271,10 @@ Here is an example code snippet for OpenSearchVectorSearch deployment:
124
271
model.predict("your prompt")
125
272
126
273
127
-
FAISS Serialization
128
-
-------------------
129
-
130
-
If your documents are not too large and you dont have a OCI OpenSearch cluster, you can use ``FAISS`` as your in-memory vector store, which can also do similarty search very efficiently. For ``FAISS``, you can just use it and deploy it as it is.
274
+
Deployment of Retrieval QA with FAISS
275
+
-------------------------------------
131
276
132
-
133
-
FAISS Deployment
134
-
----------------
135
-
136
-
Here is an example code snippet for FAISS deployment:
277
+
Here is an example code snippet for deployment of Retrieval QA using FAISS as a retriever:
137
278
138
279
.. code-block:: python3
139
280
@@ -181,8 +322,10 @@ Here is an example code snippet for FAISS deployment:
0 commit comments