Getting Error 429 from Deepgram TTS with OpenAI Realtime Voice Agent in LiveKit

[UsamaAugai]

I'm encountering a persistent 429 error when using Deepgram TTS with an OpenAI realtime voice agent via LiveKit.

Here's my configuration:

llm = openai.realtime.RealtimeModel(
    model="gpt-4o-realtime-preview-2025-06-03",
    modalities=["text"],
    temperature=0.7,
)

tts = deepgram.TTS(
    model="aura-2-andromeda-en",
    api_key=os.getenv('DEEPGRAM_API_KEY'),
    encoding='linear16',
    sample_rate=24000,
)

And this is the error I'm seeing repeatedly:

2025-07-29 13:29:14,045 - WARNING livekit.agents - failed to synthesize speech, retrying in 0.1s {"tts": "livekit.plugins.deepgram.tts.TTS", "attempt": 1, "streamed": true}
Traceback (most recent call last):
  File "C:\Users\Administrator\Desktop\voiceai\venv\Lib\site-packages\livekit\agents\tts\tts.py", line 348, in _main_task
    await self._run(output_emitter)
  File "C:\Users\Administrator\Desktop\voiceai\venv\Lib\site-packages\livekit\plugins\deepgram\tts.py", line 260, in _run
    raise APIStatusError(
livekit.agents._exceptions.APIStatusError: Invalid response status (status_code=429, request_id=03a9e3979927, body=None, retryable=False)

I'm only using a single connection to the agent, so there are no concurrent or parallel requests.

Has anyone faced this before? Is there a known rate limit or quota for Deepgram TTS that I'm possibly hitting, even with just one request at a time?

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The deepgram product you are using (e.g Speech to Text, Agent API)

[UsamaAugai]

Deepgram Text to Speech

[jkroll-deepgram]

Hi @UsamaAugai, can you please share your Deepgram project ID, so we can look into your request history?

Are you using REST or streaming TTS?

Our Aura-2 rate limits are 5 concurrent requests, so I wouldn't expect you to run into rate limits with a single request.

[UsamaAugai]

5fe8dcfa-253d-4233-98ea-e8eb73894ac6
I am using Streaming TTS with livekit

[jkroll-deepgram]

@UsamaAugai here's your Aura-2 streaming TTS concurrency over the past week. We observed a couple of spikes over the last two days, one at 12 concurrent requests and one at 16 concurrent requests, while our rate limit is 5 concurrent requests. This is why you're observing 429s. If you are not using more than a single connection through LiveKit, it's possible that under the hood the websocket is not closing immediately, or some other behavior where you're using greater concurrency than you'd think.

[UsamaAugai]

@jkroll-deepgram What would be the solution to reduce these concurrent connections or close the web sockets? Because I am just using the default code as provided by Livekit without any further configurations.

[jkroll-deepgram]

I would recommend touching base with LiveKit; they have a Slack community. Deepgram doesn't manage or define how LiveKit opens connections to our TTS endpoint. LiveKit may be able to root-cause or explain why a single LiveKit connection opens multiple Deepgram TTS connections.