docs/integrations/llms/oci_generative_ai/

[giscus[bot]]

docs/integrations/llms/oci_generative_ai/

Oracle Cloud Infrastructure Generative AI

https://python.langchain.com/docs/integrations/llms/oci_generative_ai/

[andyshaotw]

Hello,

I used the sample code with model_id="cohere.command". it doesn't work. I get the HTTP 400 and the message cohere.command not found. Could anyone have a suggestion for this?

I used the service endpoint at ap-osaka-1 not us-chicago-1.

ServiceError: {'target_service': 'generative_ai_inference', 'status': 400, 'code': '400', 'opc-request-id': 'xxxxxxxxxx', 'message': 'Entity with key cohere.command not found', 'operation_name': 'generate_text', 'timestamp': '2025-01-04T17:40:26.970305+00:00', 'client_version': 'Oracle-PythonSDK/2.141.1', 'request_endpoint': 'POST https://inference.generativeai.ap-osaka-1.oci.oraclecloud.com/20231130/actions/generateText'

Thank you!

andyshaotw

[satyasahoo210]

In sample code the model_id='cohere.command' is deprecated. You can use cohere.command-r-16k instead. Please use model_id from the official doc.
https://docs.oracle.com/en-us/iaas/Content/generative-ai/pretrained-models.htm

[bhnorman11]

Hello, I'm using the sample code but with the model_id="cohere.command-r-plus-08-2024"

Getting the following error of:

oci.exceptions.ServiceError: {'target_service': 'generative_ai_inference', 'status': 400, 'code': '400', 'opc-request-id': 'xxxxxx', 'message': 'cohere.command-r-plus-08-2024 does not support TextGeneration', 'operation_name': 'generate_text', 'timestamp': '2025-01-21T16:32:48.843187+00:00', 'client_version': 'Oracle-PythonSDK/2.143.0', 'request_endpoint': 'POST https://inference.generativeai.uk-london-1.oci.oraclecloud.com/20231130/actions/generateText'

Any ideas? Thanks!

[satyasahoo210]

Currently, TextGeneration & Summarization is deprecated and is only available with DACs. You can use ChatOCIGenAI class from langchain_community.chat_models instead of OCIGenAI

[surendra064]

With_srtructured_output is not working

[satyasahoo210]

I tried it myself and it's working fine. Here's a sample code.

from langchain_community.chat_models import ChatOCIGenAI

chat = ChatOCIGenAI(
      model_id="model_name", 
      service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
      compartment_id="compartment_id",
      model_kwargs={"temperature": 0, "max_tokens": 500},
      is_stream=False
)


from langchain_core.pydantic_v1 import BaseModel, Field

class Joke(BaseModel):
    """Joke to tell user."""
    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")

structured_chat = chat.with_structured_output(Joke)
response = structured_chat.invoke("Tell me a joke about cats")
print(response)

[surendra064]

i am still getting None

[surendra064]

ok when we are not using model_kwargs argument is is giving none any idea why by the way are u in slack?

[satyasahoo210]

Need to check.
Yes I'm in slack

[surendra064]

could u give your guid has some doubts regarding langraph ?

[satyasahoo210]

satyabrs