Skip to content

[Bug]: vertex ai service account json -unable to resolve 'environment_id' field #6141

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
vaghelarahul94 opened this issue Oct 10, 2024 · 10 comments · May be fixed by #10210
Open

[Bug]: vertex ai service account json -unable to resolve 'environment_id' field #6141

vaghelarahul94 opened this issue Oct 10, 2024 · 10 comments · May be fixed by #10210
Labels
bug Something isn't working unable to repro

Comments

@vaghelarahul94
Copy link

What happened?

When using litellm SDK version 1.48.7 like this:

from litellm import completion
import json

## GET CREDENTIALS
file_path = 'PATH_TO_JSON'

# Load the JSON file
with open(file_path, 'r') as file:
    vertex_credentials = json.load(file)

# Convert to JSON string
vertex_credentials_json = json.dumps(vertex_credentials)


response = completion(
  model="vertex_ai/gemini-pro",
  messages=[{"content": "You are a good bot.","role": "system"}, {"content": "tell me poem on pasta","role": "user"}],
  vertex_credentials=vertex_credentials_json,
  vertex_project="my_project_id",
  vertex_location="us-central1"
)

We are seeing error with stack trace:

Traceback (most recent call last):
  File "/home/appuser/.local/lib/python3.11/site-packages/litellm/main.py", line 2280, in completion
    model_response = vertex_chat_completion.completion(  # type: ignore
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/gemini/vertex_and_google_ai_studio_gemini.py", line 1208, in completion
    _auth_header, vertex_project = self._ensure_access_token(
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/vertex_llm_base.py", line 137, in _ensure_access_token
    self._credentials, cred_project_id = self.load_auth(
                                         ^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/vertex_llm_base.py", line 79, in load_auth
    creds = identity_pool.Credentials.from_info(json_obj)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/identity_pool.py", line 425, in from_info
    return super(Credentials, cls).from_info(info, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/external_account.py", line 591, in from_info
    return cls(
           ^^^^
  File "/home/appuser/.local/lib/python3.11/site-packages/google/auth/identity_pool.py", line 273, in __init__
    raise exceptions.MalformedError(
google.auth.exceptions.MalformedError: Invalid Identity Pool credential_source field 'environment_id'

Whereas when I use vertex ai SDK itself like this with my same service account credentials file like this:

import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="PATH_TO_JSON"
os.environ["VERTEXAI_LOCATION"]="us-central1"
os.environ["VERTEXAI_PROJECT"]="my_project_id""

import vertexai
from vertexai.generative_models import GenerativeModel

vertexai.init(project="my_project_id"", location="us-central1")

model = GenerativeModel("gemini-pro")

response = model.generate_content(
    "tell me poem on pasta"
)

print(response.text)

Then it works fine.

My service account file looks like this:

{
  "type": "external_account",
  "audience": "....",
  "subject_token_type": "....",
  "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/.....iam.gserviceaccount.com:generateAccessToken",
  "token_url": "https://sts.googleapis.com/v1/token",
  "credential_source": {
    "environment_id": "aws1",
    "region_url": "....",
    "url": "....",
    "regional_cred_verification_url": "...."
  }
}

Relevant log output

No response

Twitter / LinkedIn details

No response
@vaghelarahul94 vaghelarahul94 added the bug Something isn't working label Oct 10, 2024
@krrishdholakia
Copy link
Contributor

google.auth.exceptions.MalformedError: Invalid Identity Pool credential_source field 'environment_id'

this error is coming from the google sdk not litellm

os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="PATH_TO_JSON"

Try doing using the env var for litellm, and see if that works instead @vaghelarahul94

@vaghelarahul94
Copy link
Author

vaghelarahul94 commented Oct 11, 2024 •
edited
Loading

Hi @krrishdholakia Thanks for your response! I appreciate it.

I updated the code to use the os.environ["GOOGLE_APPLICATION_CREDENTIALS"]="PATH_TO_JSON" environment variable. Additionally, I commented out a few other sections.

response = completion(
  model="vertex_ai/gemini-pro",
  messages=[{"content": "You are a good bot.","role": "system"}, {"content": "tell me poem on pasta","role": "user"}],
  #vertex_credentials=vertex_credentials_json,
  #vertex_project="my_project_id",
  #vertex_location="us-central1"
)

I am now seeing that litellm SDK is asking for this role serviceUsageConsumer ? Whereas vertex ai sdk didn't ask for it. Why when using via litellm sdk this is happening?

litellm.exceptions.BadRequestError: litellm.BadRequestError: VertexAIException BadRequestError - ('Unable to acquire impersonated credentials', '{\n "error": {\n "code": 403,\n "message": "Caller does not have required permission to use project my_project_id. Grant the caller the roles/serviceusage.serviceUsageConsumer role, or a custom role with the serviceusage.services.use permission, by visiting https://console.developers.google.com/iam-admin/iam/project?project=my_project_id and then retry. Propagation of the new permission may take a few minutes.",\n "status": "PERMISSION_DENIED",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.Help",\n "links": [\n {\n "description": "Google developer console IAM admin",\n "url": "https://console.developers.google.com/iam-admin/iam/project?project=my_project_id"\n }\n ]\n },\n {\n "@type": "type.googleapis.com/google.rpc.ErrorInfo",\n "reason": "USER_PROJECT_DENIED",\n "domain": "googleapis.com",\n "metadata": {\n "service": "iamcredentials.googleapis.com",\n "consumer": "projects/my_project_id"\n }\n }\n ]\n }\n}\n')

@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

@pritkudale
Copy link

Facing the same issue

@sammcj
Copy link
Contributor

sammcj commented Mar 14, 2025

Also have this issue, GCP and the SA is correctly configured.

@krrishdholakia
Copy link
Contributor

Unable to repro, when using vertex ai service account uploaded via litellm ui (this does the same thing of converting to a json str)

@krrishdholakia
Copy link
Contributor 

{
  "type": "service_account",
  "project_id": "my-id",
  "private_key_id": "",
  "private_key": "",
  "client_email": "",
  "client_id": "",
  "auth_uri": "",
  "token_uri": "",
  "auth_provider_x509_cert_url": "",
  "client_x509_cert_url": "",
  "universe_domain": "googleapis.com"
}

this is my service account structure

@krrishdholakia krrishdholakia changed the title [Bug]: litellm not working with vertex ai when using service account json file [Bug]: vertex ai service account json -unable to resolve 'environment_id' field Mar 14, 2025
@krrishdholakia
Copy link
Contributor

How can i create a similar service account to yours? @sammcj @vaghelarahul94

@sammcj
Copy link
Contributor

sammcj commented Mar 14, 2025

Sure thing, here's my json:

{
  "type": "service_account",
  "project_id": "redacted",
  "private_key_id": "redacted",
  "private_key": "redacted",
  "client_email": "samm-vertex-sa@redacted.iam.gserviceaccount.com",
  "client_id": "redacted",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/samm-vertex-sa%40redacted.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}

The SA has access to the usual list of VertexAI services.

litellm config:

...
  - model_name: gemini-2.0-pro-exp-02-05
    litellm_params:
      model: vertex_ai/gemini-2.0-pro-exp-02-05
      vertex_project: redacted
      vertex_location: us-east-5

@mrT23
Copy link

mrT23 commented Apr 20, 2025

@sammcj
I struggled with similar problems.
what worked for me is direct injection. i think the litellm_params is not working for vertex_ai

import ujson
from litellm import completion
import json



# Convert to JSON string
vertex_credentials_json = load(...)

response = completion(
  model="vertex_ai/gemini-2.5-pro-preview-03-25",
  messages=[{"content": "You are a good bot.","role": "system"}, {"content": "tell me poem on pasta","role": "user"}],
  vertex_credentials=vertex_credentials_json,
)
print(response)

@pascallim pascallim linked a pull request Apr 22, 2025 that will close this issue
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working unable to repro
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: add workload identity federation between GCP and AWS pascallim/litellm
5 participants
@sammcj @krrishdholakia @vaghelarahul94 @mrT23 @pritkudale