Feature request: Vertex AI Function Calling for FunctionCallingAgentWorker

[wadave]

After you crate a vertex model:

from llama_index.llms.vertex import Vertex
vertex_gemini = Vertex(model="gemini-1.5-pro", temperature=0, additional_kwargs={})
vertex_gemini.metadata

the model metadata shows:

is_function_calling_model=False

This makes

from llama_index.core.agent import AgentRunner

agent_worker = FunctionCallingAgentWorker.from_tools(
    [vector_tool, summary_tool],
    llm=vertex_gemini,
    verbose=True
)
agent = AgentRunner(agent_worker)

failed:

ValueError: Model name gemini-1.5-pro does not support function calling API. 

Is there a plan to support Vertex AI Function Calling with Llamaindex?

[ravi03071991]

Hey @wadave ,

The Google Vertex AI model is not yet integrated with FunctionCalling. Are you interested in raising a PR for this integration?

[wadave]

Hi @ravi03071991 I created a PR for this:

#14088