.Net: Introduce support for response modalities and audio options in AzureClientCore
#12523
+481
−4
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduce methods to handle response modalities and audio options in AzureClientCore. Add checks for executionSettings.Modalities and executionSettings.Audio to dynamically configure options based on user settings. Implement GetResponseModalities and GetAudioOptions methods to support various input formats, improving flexibility and robustness.
markwallace-microsoft
added
.NET
github-actions
bot
changed the title
AzureClientCoreAzureClientCore
|
@microsoft-github-policy-service agree |
|
This is for this request #11720 |
|
@Cobra86 Thank you for adding the support, to get those one in, we also need Unit Tests similar how we do have for OpenAI Connector. Please add those. Overall LGTM. |
Implemented tests in `AzureOpenAIChatCompletionServiceTests` to verify the correct handling of audio content in requests and responses. This includes checks for sending audio content, processing audio responses, and handling audio metadata. Introduced new theory data members for validating response modalities and audio options.
|
@RogerBarreto Thank you :). I've added the tests same as OpenAI. Please let me know if i need more tests |
RogerBarreto
approved these changes
Jun 19, 2025
westey-m
reviewed
Jun 20, 2025
dotnet/src/Connectors/Connectors.AzureOpenAI/Core/AzureClientCore.ChatCompletion.cs
Outdated
Show resolved
Hide resolved
westey-m
reviewed
Jun 20, 2025
dotnet/src/Connectors/Connectors.AzureOpenAI/Core/AzureClientCore.ChatCompletion.cs
Outdated
Show resolved
Hide resolved
westey-m
reviewed
Jun 20, 2025
dotnet/src/Connectors/Connectors.AzureOpenAI/Core/AzureClientCore.ChatCompletion.cs
Outdated
Show resolved
Hide resolved
westey-m
requested changes
Jun 20, 2025
Enhance error handling by introducing try-catch blocks for JSON deserialization, providing clearer exception messages for unsupported modalities and invalid audio options. Refactor parsing logic for string modalities to improve code readability and maintainability.
Thanks. I've made the changes. Please review it and let me know. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
This change extends
AzureClientCoreto handle response modalities and audio options dynamically based on user-providedexecutionSettings.Why this is needed:
Currently, the OpenAI connector supports audio modalities through the
ModalitiesandAudioproperties inOpenAIPromptExecutionSettings, but the Azure OpenAI connector doesn't fully implement this functionality.The code for handling audio exists in
AzureClientCore.csbut isn't included inAzureClientCore.ChatCompletion.cs.Description
Introduced
GetResponseModalitiesandGetAudioOptionshelper methods. These follow the same logic as the equivalent methods in the OpenAI connector to ensure consistent behaviour and reduce duplication across both clients.Updated
CreateChatCompletionOptionsto:ResponseModalitiesif specified inexecutionSettings.AudioOptionsif specified inexecutionSettings.Contribution Checklist