Skip to content

Commit e366c28

Browse files
authored
Updated ADS docs (#1022)
2 parents ce4a5bf + 4751824 commit e366c28

File tree

2 files changed

+32
-18
lines changed

2 files changed

+32
-18
lines changed

docs/source/user_guide/large_language_model/langchain_models.rst

Lines changed: 22 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -26,47 +26,50 @@ By default, the integration uses the same authentication method configured with
2626
.. code-block:: python3
2727
2828
import ads
29-
from ads.llm import ChatOCIModelDeploymentVLLM
29+
from ads.llm import ChatOCIModelDeployment
3030
3131
ads.set_auth(auth="resource_principal")
3232
33-
llm = ChatOCIModelDeploymentVLLM(
34-
model="odsc-llm",
33+
llm = ChatOCIModelDeployment(
34+
model="odsc-llm", # default model name if deployed on AQUA
3535
endpoint= f"https://modeldeployment.oci.customer-oci.com/<OCID>/predict",
36-
# Optionally you can specify additional keyword arguments for the model, e.g. temperature.
36+
# Optionally you can specify additional keyword arguments for the model, e.g. temperature and default_headers.
3737
temperature=0.1,
38+
default_headers={"route": "v1/chat/completions"}, # default route for chat models
3839
)
3940
4041
Alternatively, you may use specific authentication for the model:
4142

4243
.. code-block:: python3
4344
4445
import ads
45-
from ads.llm import ChatOCIModelDeploymentVLLM
46+
from ads.llm import ChatOCIModelDeployment
4647
47-
llm = ChatOCIModelDeploymentVLLM(
48-
model="odsc-llm",
48+
llm = ChatOCIModelDeployment(
49+
model="odsc-llm", # default model name if deployed on AQUA
4950
endpoint= f"https://modeldeployment.oci.customer-oci.com/<OCID>/predict",
5051
# Use security token authentication for the model
5152
auth=ads.auth.security_token(profile="my_profile"),
52-
# Optionally you can specify additional keyword arguments for the model, e.g. temperature.
53+
# Optionally you can specify additional keyword arguments for the model, e.g. temperature and default_headers.
5354
temperature=0.1,
55+
default_headers={"route": "v1/chat/completions"}, # default route for chat models
5456
)
5557
5658
Completion Models
5759
=================
5860

59-
Completion models takes a text string and input and returns a string with completions. To use completion models, your model should be deployed with the completion endpoint (``/v1/completions``). The following example shows how you can use the ``OCIModelDeploymentVLLM`` class for model deployed with vLLM container. If you deployed the model with TGI container, you can use ``OCIModelDeploymentTGI`` similarly.
61+
Completion models takes a text string and input and returns a string with completions. To use completion models, your model should be deployed with the completion endpoint (``/v1/completions``).
6062

6163
.. code-block:: python3
6264
63-
from ads.llm import OCIModelDeploymentVLLM
65+
from ads.llm import OCIModelDeploymentLLM
6466
65-
llm = OCIModelDeploymentVLLM(
66-
model="odsc-llm",
67+
llm = OCIModelDeploymentLLM(
68+
model="odsc-llm", # default model name if deployed on AQUA
6769
endpoint= f"https://modeldeployment.oci.customer-oci.com/<OCID>/predict",
6870
# Optionally you can specify additional keyword arguments for the model.
6971
max_tokens=32,
72+
default_headers={"route": "v1/completions"}, # default route for completion models
7073
)
7174
7275
# Invoke the LLM. The completion will be a string.
@@ -87,18 +90,19 @@ Completion models takes a text string and input and returns a string with comple
8790
Chat Models
8891
===========
8992

90-
Chat models takes `chat messages <https://python.langchain.com/docs/concepts/#messages>`_ as inputs and returns additional chat message (usually `AIMessage <https://python.langchain.com/docs/concepts/#aimessage>`_) as output. To use chat models, your models must be deployed with chat completion endpoint (``/v1/chat/completions``). The following example shows how you can use the ``ChatOCIModelDeploymentVLLM`` class for model deployed with vLLM container. If you deployed the model with TGI container, you can use ``ChatOCIModelDeploymentTGI`` similarly.
93+
Chat models takes `chat messages <https://python.langchain.com/docs/concepts/#messages>`_ as inputs and returns additional chat message (usually `AIMessage <https://python.langchain.com/docs/concepts/#aimessage>`_) as output. To use chat models, your models must be deployed with chat completion endpoint (``/v1/chat/completions``).
9194

9295
.. code-block:: python3
9396
9497
from langchain_core.messages import HumanMessage, SystemMessage
95-
from ads.llm import ChatOCIModelDeploymentVLLM
98+
from ads.llm import ChatOCIModelDeployment
9699
97-
llm = ChatOCIModelDeploymentVLLM(
98-
model="odsc-llm",
99-
endpoint=f"<oci_model_deployment_url>>/predict",
100+
llm = ChatOCIModelDeployment(
101+
model="odsc-llm", # default model name if deployed on AQUA
102+
endpoint=f"<oci_model_deployment_url>/predict",
100103
# Optionally you can specify additional keyword arguments for the model.
101104
max_tokens=32,
105+
default_headers={"route": "v1/chat/completions"}, # default route for chat models
102106
)
103107
104108
messages = [
@@ -133,7 +137,7 @@ The vLLM container support `tool/function calling <https://docs.vllm.ai/en/lates
133137
from ads.llm import ChatOCIModelDeploymentVLLM, ChatTemplates
134138
135139
llm = ChatOCIModelDeploymentVLLM(
136-
model="odsc-llm",
140+
model="odsc-llm", # default model name if deployed on AQUA
137141
endpoint= f"https://modeldeployment.oci.customer-oci.com/<OCID>/predict",
138142
# Set tool_choice to "auto" to enable tool/function calling.
139143
tool_choice="auto",

docs/source/user_guide/model_registration/large_language_model.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22
Large Language Model
33
====================
44

5+
.. admonition:: Introducing AI Quick Actions
6+
:class: note
7+
8+
Oracle Data Science now has launched `AI Quick Actions <https://docs.oracle.com/en-us/iaas/data-science/using/ai-quick-actions.htm>`_, which makes it easy for you to browse foundation models, and deploy, fine-tune, and evaluate them inside Data Science notebooks.
9+
10+
.. admonition:: Use AI Quick Actions
11+
:class: note
12+
13+
The following page is obsolete and please use `AI Quick Actions <https://docs.oracle.com/en-us/iaas/data-science/using/ai-quick-actions.htm>`_ instead.
14+
515
Oracle ADS (Accelerated Data Science) opens the gateway to harnessing the full potential of the Large Language models
616
within Oracle Cloud Infrastructure (OCI). `Meta <https://ai.meta.com/resources/models-and-libraries/llama-downloads/>`_'s
717
latest offering, `Llama 2 <https://ai.meta.com/llama/>`_, introduces a collection of pre-trained and

0 commit comments

Comments
 (0)