Claude on Vertex AI Model Garden

[WaterKnight1998]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Google added Claude model family to Vertex AI Garden but is not availabe inside langchain SDK.

I tried using like this

import json
import os

import vertexai
from google.oauth2.service_account import Credentials
from langchain_google_vertexai.model_garden import VertexAIModelGarden

GOOGLE_CREDENTIALS = Credentials.from_service_account_info(
    json.loads(os.getenv("GCP_LIC"), strict=False)
)
GCP_PROJECT_ID = os.getenv("GCP_PROJECT_ID")
GCP_REGION = os.getenv("GCP_REGION")

vertexai.init(
    project=GCP_PROJECT_ID,
    location=GCP_REGION,
    credentials=GOOGLE_CREDENTIALS,
)

llm_model = VertexAIModelGarden(
                    project=GCP_PROJECT_ID,
                    endpoint_id="anthropic",
                    model_name="claude-3-opus@20240229",
                    convert_system_message_to_human=True,
                    temperature=temperature,
                )

Error:

Error calling LLM Model CLAUDE_3_OPUS_v1: 404 Endpoint `projects/1234435645/locations/us-central1/endpoints/anthropic` not found.

Endpoint should be: projects/PROJECT_ID/locations/LOCATION/publishers/anthropic/models/MODEL:streamRawPredict

More info here

Motivation

Not support right now

Proposal (If applicable)

Add publisher to VertexAIModelGarden

[henriques-luiz]

any updates on that? I'm trying to use mistral on garden but it seems impossible to make it work :(
I'm also trying to use as
return ChatVertexAI( model_name=model.identifier, max_output_tokens=max_tokens, temperature=temperature, project=vertexai_project, location=vertexai_location, api_transport="rest", #api_endpoint=vertexai_location+"-aiplatform.googleapis.com", )

and I'm getting

google.api_core.exceptions.InvalidArgument: 400 Invalid resource field value in the request. [reason: "RESOURCE_PROJECT_INVALID" domain: "googleapis.com" metadata { key: "service" value: "aiplatform.googleapis.com" } metadata { key: "method" value: "google.cloud.aiplatform.v1beta1.PredictionService.StreamGenerateContent" } ]

[dheerajiiitv]

@WaterKnight1998 It might not be available in us-central1 try us-east5 or europe-west1

[nick-youngblut]

I tried modifying the path via:

class AnthropicChatVertexAI(ChatVertexAI):
    @property
    def _model_endpoint(self):
        return f"projects/{self.project_id}/locations/{self.location}/publishers/anthropic/models/{self.model_name}"

...but not luck. Why must GCP products always be such a pain to use?