Regression: Vertex AI Gemini generateContent w/ audio input doesn't work

[Blickwinkel1107]

Description

Hi, this is regarding issue #14745. The issue still persist. I'm using the latest package 11.12.0. However I can still see that response json missing tokenCount field sometimes.
Example output:

11.12.0 - [FirebaseVertexAI][I-VTX003000] JSON response: {
  "candidates": [
    {
      "content": {
        "role": "model",
        "parts": [
          {
            "text": "{\n  \"has_animal\": false,\n  \"pets\": []\n}"
          }
        ]
      },
      "finishReason": "STOP",
      "avgLogprobs": -1.809538419668873e-05
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 3008,
    "candidatesTokenCount": 18,
    "totalTokenCount": 3026,
    "trafficType": "ON_DEMAND",
    "promptTokensDetails": [
      {
        "modality": "IMAGE",
        "tokenCount": 2322
      },
      {
        "modality": "AUDIO"
      },
      {
        "modality": "TEXT",
        "tokenCount": 686
      }
    ],
    "candidatesTokensDetails": [
      {
        "modality": "TEXT",
        "tokenCount": 18
      }
    ]
  },
  "modelVersion": "gemini-2.0-flash-lite",
  "createTime": "2025-05-20T04:50:27.060206Z",
  "responseId": "EwosaK7WA_-MhMIPmdvrOQ"
}

Error message:

Error sending to Gemini: internalError(underlying: Swift.DecodingError.keyNotFound(CodingKeys(stringValue: "tokenCount", intValue: nil), Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "usageMetadata", intValue: nil), CodingKeys(stringValue: "promptTokensDetails", intValue: nil), _CodingKey(stringValue: "Index 1", intValue: 1)], debugDescription: "No value associated with key CodingKeys(stringValue: \"tokenCount\", intValue: nil) (\"tokenCount\").", underlyingError: nil)))

Reproducing the issue

No response

Firebase SDK Version

11.12.0

Xcode Version

16.3

Installation Method

Swift Package Manager

Firebase Product(s)

VertexAI

Targeted Platforms

iOS

Relevant Log Output

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

[andrewheard]

Hi @Blickwinkel1107, we have a release going out today 11.13.0 that will include the fix #14747. Sorry for the delay in getting 11.13.0 released, our usual schedule was changed due to Google I/O.

[andrewheard]

The Firebase 11.13.0 release is now available. I'll close this issue but feel free to re-open if you're still running into the problem after upgrading.

Note: You may also want to migrate to the Firebase AI Logic SDK (migration guide), which is replacing Vertex AI in Firebase. See the release notes for more details.