Feature Request: Vertex AI Service Account Support and VertexAIBatch Inference

[andrediaspx]

I'm opening this issue to request a couple of features that would improve integration with VertexAI and help reduce GCP usage costs.

1. Support for Vertex AI Service Accounts

It appears the library currently requires API KEY for authentication. It would be very helpful if it could also support authentication via Vertex AI service accounts json file.

2. Support for Batch Inference in VertexAI

When processing a large number of documents, making a separate API call for each one becomes expensive. A more cost-effective approach is to use batching.

It would be great if the library could be updated to internally collect multiple inputs and send them to the Gemini API as a single, batched request. This would significantly cut down on the total number of API calls, leading to substantial cost savings for anyone doing large-scale processing.

[aksg87]

Hey @codekiln, Vertex AI support is being discussed in #99 where we're consolidating provider conversations.

The new provider registry (#97) enables community plugins - a langextract-vertex package could handle both service accounts and batch inference. See examples/custom_provider_plugin/ for implementation.

Let's continue in #99 - I'll close this issue to consolidate provider discussions there.

[aksg87]

Closing to consolidate provider discussions in #99

[Shreyansh1311]

Hi @aksg87,

Thanks for the amazing work! Was support for Batch Inference added? It would be really useful when processing millions of documents.