Does using third party TTS providers degrade tool calling capabilities?

[Matin-Massoudi]

In comparison between two identical agent configurations (identical system prompts, LLM, etc.), with one utilizing Deepgram as the text-to-speech (TTS) engine and the other configured to use Elevenlabs, the latter encounters difficulties in making tool calls unless the user explicitly commands “Invoke tool x.” Conversely, the former agent demonstrates the ability to infer more broadly from user input.

[deepgram-community[bot]]

Thanks for asking your question. Please be sure to reply with as much detail as possible so the community can assist you efficiently.
_{Consider joining our Discord community for more opportunity to engage with your fellow Deepgram users. You can earn points which can be redeemed for cool stuff by being active in our communities!}

[deepgram-community[bot]]

It looks like we're missing some important information to help debug your issue. Would you mind providing us with the following details in a reply?

• The programming language you are working in (e.g. JavaScript, Python).
• A request ID that triggered your error or issue.

[deepgram-community[bot]]

It seems like you’re comparing the performance of two agent configurations using different text-to-speech (TTS) engines, Deepgram and Elevenlabs, focusing on their ability to make tool calls based on user input. Here are some considerations:

1. Inference Capability:

   • Deepgram Configuration: Shows enhanced ability to understand and infer user needs from inputs without needing explicit commands for making tool calls.
   • Elevenlabs Configuration: Requires explicit instructions from users to make tool calls, indicating a potential difference in how inference is handled or perhaps integrated with the language model.

2. Potential Reasons:

   • Integration and Compatibility: The way Deepgram is integrated might offer better compatibility or synergy with the other components of the agent, resulting in better understanding.
   • System Prompt Optimization: It’s possible that the system prompts or the overall configuration is more tuned for the inference capabilities of the Deepgram setup.

3. Considerations for Improvement:

   • Review how each TTS engine processes and interacts with input.
   • Check any differences in implementation details that might affect how commands are interpreted.
   • Ensure that both setups operate using the same logic or pathway for inferring commands from users.

If you have more specific questions about this comparison, feel free to ask!

This message was sent by John Vajda from Deepgram, via our community automation.