Added doc for deploying langchain application. (#464)

Author: lu-ohai

docs/source/index.rst

@@ -68,6 +68,7 @@ Oracle Accelerated Data Science (ADS)
    user_guide/logs/logs
    user_guide/pipeline/index
    user_guide/secrets/index
+   user_guide/large_language_model/index
 
 .. toctree::
    :hidden:

docs/source/user_guide/large_language_model/deploy_langchain_application.rst

@@ -0,0 +1,151 @@
+############################
+Deploy LangChain Application
+############################
+
+Oracle ADS SDK now supports the deployment of LangChain application to OCI data science model deployment and you can easily do so just by writing a couple lines of code.
+
+.. versionadded:: 2.9.1
+
+Configuration
+*************
+
+Ensure that you have created the necessary `policies, authentication, and authorization for model deployments <https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-policies-auth.htm#model_dep_policies_auth>`_. 
+Here we're using the ``resource_principal`` as auth type and you can configure the policy as below.
+
+.. code-block:: shell
+
+    allow dynamic-group <dynamic-group-name> to manage data-science-model-deployments in compartment <compartment-name>
+
+Create LangChain Application
+****************************
+
+Create a simple LangChain application that links prompt and cohere model as below. Remember to replace the ``<cohere_api_key>`` with the actual cohere api key.
+
+.. code-block:: python3
+
+    import os
+    from langchain.llms import Cohere
+    from langchain.chains import LLMChain
+    from langchain.prompts import PromptTemplate
+    os.environ["COHERE_API_KEY"] = "<cohere_api_key>"
+    
+    cohere = Cohere()
+    prompt = PromptTemplate.from_template("Tell me a joke about {subject}")
+    llm_chain = LLMChain(prompt=prompt, llm=cohere, verbose=True)
+
+Now you have a LangChain object ``llm_chain``. Try running it with the prompt ``{"subject": "animals"}`` and it should give you the corresponding result.
+
+.. code-block:: python3
+
+    llm_chain.run({"subject": "animals"})
+
+Initialize the ChainDeployment
+******************************
+
+Initialize class ``ChainDeployment`` from ADS SDK and pass the LangChain object ``llm_chain`` from previous step as parameter.
+The ``artifact_dir`` is an optional parameter which points to the folder where the model artifacts will be put locally. 
+In this example, we're using a temporary folder generated by ``tempfile``.
+
+.. code-block:: python3
+
+    import tempfile
+    from ads.llm.deploy import ChainDeployment
+    
+    artifact_dir = tempfile.mkdtemp()
+    
+    chain_deployment = ChainDeployment(
+        chain=llm_chain,
+        artifact_dir=artifact_dir
+    )
+
+Prepare the Model Artifacts
+***************************
+
+Call ``prepare`` from ``ChainDeployment`` to generate the ``score.py`` and serialize the LangChain application to ``chain.yaml`` file under ``artifact_dir`` folder. 
+Parameters ``inference_conda_env`` and ``inference_python_version`` are passed to define the conda environment where your LangChain application will be running on OCI cloud. 
+Here we're using ``pytorch21_p39_gpu_v1`` with python 3.9.
+
+.. code-block:: python3
+
+    chain_deployment.prepare(
+        inference_conda_env="pytorch21_p39_gpu_v1",
+        inference_python_version="3.9",
+    )
+
+Below is the ``chain.yaml`` file that was saved from ``llm_chain`` object. For more information regarding LLMs model serialization, see `here <https://python.langchain.com/docs/modules/model_io/llms/llm_serialization>`_.
+
+.. code-block:: YAML
+    
+    _type: llm_chain
+    llm:
+      _type: cohere
+      frequency_penalty: 0.0
+      k: 0
+      max_tokens: 256
+      model: null
+      p: 1
+      presence_penalty: 0.0
+      temperature: 0.75
+      truncate: null
+    llm_kwargs: {}
+    memory: null
+    metadata: null
+    output_key: text
+    output_parser:
+      _type: default
+    prompt:
+      _type: prompt
+      input_types: {}
+      input_variables:
+      - subject
+      output_parser: null
+      partial_variables: {}
+      template: Tell me a joke about {subject}
+      template_format: f-string
+      validate_template: false
+    return_final_only: true
+    tags: null
+    verbose: true
+
+Save Artifacts to OCI Model Catalog
+***********************************
+
+Call ``save`` to pack and upload the artifacts under ``artifact_dir`` to OCI data science model catalog. Once the artifacts are successfully uploaded, you should be able to see the id of the model.
+
+.. code-block:: python3
+
+    chain_deployment.save(display_name="LangChain Model")
+
+Deploy the Model
+****************
+
+Deploy the LangChain model from previous step by calling ``deploy``. Remember to replace the ``<cohere_api_key>`` with the actual cohere api key in the ``environment_variables``. 
+It usually takes a couple of minutes to deploy the model and you should see the model deployment in the output once the process completes.
+
+.. code-block:: python3
+
+    chain_deployment.deploy(
+        display_name="LangChain Model Deployment",
+        environment_variables={"COHERE_API_KEY":"<cohere_api_key>"},
+    )
+
+Invoke the Deployed Model
+*************************
+
+Now the OCI data science model deployment endpoint is ready and you can invoke it to ``tell a joke about animals``.
+
+.. code-block:: python3
+
+    chain_deployment.predict(data={"subject": "animals"})["output"]
+
+.. figure:: figures/prediction.png
+  :width: 800
+
+Alternatively, you can use OCI CLI to invoke the model deployment. Remember to replace the ``langchain_application_model_deployment_url`` with the actual model deployment url which you can find in the output from deploy step.
+
+.. code-block:: shell
+
+    oci raw-request --http-method POST --target-uri <langchain_application_model_deployment_url>/predict --request-body '{"subject": "animals"}' --auth resource_principal
+
+.. figure:: figures/cli_prediction.png
+  :width: 800

docs/source/user_guide/large_language_model/figures/cli_prediction.png

@@ -0,0 +1,12 @@
+.. _large_language_model:
+
+####################
+Large Language Model
+####################
+
+
+.. toctree::
+    :hidden:
+    :maxdepth: 2
+
+    deploy_langchain_application

docs/source/user_guide/large_language_model/figures/prediction.png

@@ -0,0 +1,63 @@
+import os
+import tempfile
+from unittest.mock import MagicMock, patch
+
+from ads.llm.deploy import ChainDeployment
+
+from langchain.chains import LLMChain
+from langchain.prompts import PromptTemplate
+from tests.unitary.with_extras.langchain.test_guardrails import FakeLLM
+
+class TestLangChainDeploy:
+    
+    def generate_chain_application(self):
+        prompt = PromptTemplate.from_template("Tell me a joke about {subject}")
+        llm_chain = LLMChain(prompt=prompt, llm=FakeLLM(), verbose=True)
+        return llm_chain
+    
+    @patch("ads.model.datascience_model.DataScienceModel._to_oci_dsc_model")
+    def test_initialize(self, mock_to_oci_dsc_model):
+        chain_application = self.generate_chain_application()
+        chain_deployment = ChainDeployment(chain_application, auth=MagicMock())
+        mock_to_oci_dsc_model.assert_called()
+        assert isinstance(chain_deployment.chain, LLMChain)
+
+    @patch("ads.model.runtime.env_info.get_service_packs")
+    @patch("ads.common.auth.default_signer")
+    @patch("ads.model.datascience_model.DataScienceModel._to_oci_dsc_model")
+    def test_prepare(self, mock_to_oci_dsc_model, mock_default_signer, mock_get_service_packs):
+        mock_default_signer.return_value = MagicMock()
+        inference_conda_env="oci://service-conda-packs@ociodscdev/service_pack/cpu/General_Machine_Learning_for_CPUs/1.0/mlcpuv1"
+        inference_python_version="3.7"
+        mock_get_service_packs.return_value = (
+            {
+                inference_conda_env : ("mlcpuv1", inference_python_version),
+            },
+            {
+                "mlcpuv1" : (inference_conda_env, inference_python_version),
+            }
+        )
+        artifact_dir = tempfile.mkdtemp()
+        chain_application = self.generate_chain_application()
+        chain_deployment = ChainDeployment(
+            chain_application, 
+            artifact_dir=artifact_dir, 
+            auth=MagicMock()
+        )
+        
+        mock_to_oci_dsc_model.assert_called()
+        mock_default_signer.assert_called()
+
+        chain_deployment.prepare(
+            inference_conda_env="oci://service-conda-packs@ociodscdev/service_pack/cpu/General_Machine_Learning_for_CPUs/1.0/mlcpuv1",
+            inference_python_version="3.7"
+        )
+
+        mock_get_service_packs.assert_called()
+
+        score_py_file_location = os.path.join(chain_deployment.artifact_dir, "score.py")
+        chain_yaml_file_location = os.path.join(chain_deployment.artifact_dir, "chain.yaml")
+        runtime_yaml_file_location = os.path.join(chain_deployment.artifact_dir, "runtime.yaml")
+        assert os.path.isfile(score_py_file_location) and os.path.getsize(score_py_file_location) > 0
+        assert os.path.isfile(chain_yaml_file_location) and os.path.getsize(chain_yaml_file_location) > 0
+        assert os.path.isfile(runtime_yaml_file_location) and os.path.getsize(runtime_yaml_file_location) > 0