Skip to content

Notebook and sample data for BugBash of IntentResolution, TaskAdherence, Relevance Evaluators v2 #42012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: users/singankit/agent_evaluators_v2_bug_bash
Choose a base branch
from
+625 −108
Open

Notebook and sample data for BugBash of IntentResolution, TaskAdherence, Relevance Evaluators v2 #42012

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9d7aea0
Made response_format option type public and update docstr (#41991)
howieleung Jul 14, 2025
2f365a6
Notebook and sample data for BugBash of IntentResolution, TaskAdheren…
JoseCSantos Jul 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion sdk/ai/azure-ai-agents/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

# Release History

## 1.1.0b5 (Unreleased)

### Bugs Fixed

- `AgentsResponseFormatOption`, `MessageInputContent`, `MessageAttachmentToolDefinition`, `AgentsToolChoiceOption` are now public.

## 1.1.0b4 (2025-07-11)

### Features Added
Expand All @@ -12,7 +18,7 @@ the thread. Default value is None.

### Bugs Fixed

- `_AgentsClientOperationsMixin` now it is private.
- `_AgentsClientOperationsMixin` is now private.

### Sample updates

Expand Down
74 changes: 37 additions & 37 deletions sdk/ai/azure-ai-agents/azure/ai/agents/_patch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
from . import models as _models
from ._client import AgentsClient as AgentsClientGenerated
from .operations._patch import _has_errors_in_toolcalls_output
from . import _types
from . import types as _types

if sys.version_info >= (3, 9):
from collections.abc import MutableMapping
Expand Down Expand Up @@ -140,10 +140,10 @@ def create_agent( # pylint: disable=arguments-differ
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword metadata: Custom key-value pairs for storing additional information about the agent.
Useful for categorization, tracking, or storing application-specific data. Limited to 16 pairs,
with keys up to 64 characters and values up to 512 characters each.
Expand Down Expand Up @@ -205,10 +205,10 @@ def create_agent( # pylint: disable=arguments-differ
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword metadata: Custom key-value pairs for storing additional information about the agent.
Useful for categorization, tracking, or storing application-specific data. Limited to 16 pairs,
with keys up to 64 characters and values up to 512 characters each.
Expand Down Expand Up @@ -295,10 +295,10 @@ def create_agent(
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword metadata: Custom key-value pairs for storing additional information about the agent.
:paramtype metadata: Optional[Dict[str, str]]
:keyword content_type: The MIME type of the request body when using body parameter.
Expand Down Expand Up @@ -392,10 +392,10 @@ def update_agent( # pylint: disable=arguments-differ
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword metadata: A set of up to 16 key/value pairs that can be attached to an object, used
for storing additional information about that object in a structured format. Keys may be up to
64 characters in length and values may be up to 512 characters in length. Default value is
Expand Down Expand Up @@ -458,10 +458,10 @@ def update_agent( # pylint: disable=arguments-differ
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword metadata: A set of up to 16 key/value pairs that can be attached to an object, used
for storing additional information about that object in a structured format. Keys may be up to
64 characters in length and values may be up to 512 characters in length. Default value is
Expand Down Expand Up @@ -569,10 +569,10 @@ def update_agent(
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword content_type: Body Parameter content-type. Content type parameter for JSON body.
Default value is "application/json".
:paramtype content_type: str
Expand Down Expand Up @@ -732,10 +732,10 @@ def create_thread_and_run(
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword parallel_tool_calls: If True, tools will be invoked in parallel.
:type parallel_tool_calls: bool
:keyword metadata: Up to 16 key/value pairs for structured metadata on the run.
Expand Down Expand Up @@ -836,10 +836,10 @@ def create_thread_and_run( # type: ignore
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword parallel_tool_calls: If True, tools will be invoked in parallel.
:type parallel_tool_calls: bool
:keyword metadata: Up to 16 key/value pairs for structured metadata on the run.
Expand Down Expand Up @@ -938,10 +938,10 @@ def create_thread_and_process_run(
:keyword response_format: Specifies the format for the responses, particularly for tool calls.
Can be a string, response format mode, or structured response format object that defines how
the agent should structure its outputs.
:type response_format: str or
~azure.ai.agents.models.AgentsApiResponseFormatMode or
~azure.ai.agents.models.AgentsApiResponseFormat or
~azure.ai.agents.models.ResponseFormatJsonSchemaType
:type response_format: Optional[Union[str,
~azure.ai.agents.models.AgentsApiResponseFormatMode,
~azure.ai.agents.models.AgentsApiResponseFormat,
~azure.ai.agents.models.ResponseFormatJsonSchemaType]]
:keyword parallel_tool_calls: If True, allows tool calls to be executed in parallel.
:type parallel_tool_calls: bool, optional
:keyword metadata: Optional metadata (up to 16 key/value pairs) to attach to the run.
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-agents/azure/ai/agents/_version.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

VERSION = "1.1.0b4"
VERSION = "1.1.0b5"
Loading