Agents - Cannot use model deployments from connected resources?

[nrobert]

Hi,

When I try to create an Agent in my project (with the GA version in an Azure AI Foundry project, not a hub-based project), I can only use model deployments deployed in my current Azure AI Foundry resource, despite having connected Azure OpenAI resources.
=> Is it normal?

Screenshots:
I can see the deployments:

I can use them in the Chat playground:

As you can see in my agent setup, I only have 1 deployment (the one from my Azure AI Foundry resource):

[mukeshagmsft]

As of now there's no easy way to do this. However, this is a way.

1. The connected OpenAI resource should be in the same region as the AIServices resource.
2. You would need to create capability host. This could disrupt any existing agents under the AIServices resource.

How to create capability host?
Below is a python code - replace everywhere you see {something}.

import requests
import json
from azure.identity import DefaultAzureCredential

token = DefaultAzureCredential().get_token("https://management.azure.com/.default").token 
 
 
 ## Create account capability host
url = "https://management.azure.com/subscriptions/{sub-id}/resourceGroups/{res-group-name}/providers/Microsoft.CognitiveServices/accounts/{ai-services-resource}/capabilityHosts/{any-cap-host-name}/"

querystring = {"api-version":"2025-04-01-preview"}

payload = {"properties": {"capabilityHostKind": "Agents"}}
headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json",
    "User-Agent": "insomnia/11.0.1"
}

response = requests.request("PUT", url, json=payload, headers=headers, params=querystring)

print(response.text)


## Create project capability host

url = "https://management.azure.com/subscriptions/{sub-id}/resourceGroups/{res-group-name}/providers/Microsoft.CognitiveServices/accounts/{ai-services-resource}/projects/{project-name}/capabilityHosts/{any-proj-cap-host-name}/"

querystring = {"api-version":"2025-04-01-preview"}

payload = {"properties": {
        "capabilityHostKind": "Agents",
        "aiServicesConnections": ["{connection-name-of-aoai-resource}"]
    }}
headers = {
    "Authorization": f"Bearer {token}",
    "Content-Type": "application/json",
    "User-Agent": "insomnia/11.0.1"
}

response = requests.request("PUT", url, json=payload, headers=headers, params=querystring)

print(response.text)

The above would let you use Agents ONLY with the connected AOAI resource mentioned in the "aiServicesConnections" parameter. You could update or delete the capability host to be able to use deployments from other AOAI resource connection or from the parent AIServices resource.

[graemefoster]

@mukeshagmsft - you can change the URL of an endpoint on an AOAI connection in AI Foundry. If I connect an AOAI resource, but change it's endpoint to APIm AI Gateway, would this technique let me use an existing APIm AI Gateway for my AI Foundry Agents?

[mukeshagmsft]

@mukeshagmsft - you can change the URL of an endpoint on an AOAI connection in AI Foundry. If I connect an AOAI resource, but change it's endpoint to APIm AI Gateway, would this technique let me use an existing APIm AI Gateway for my AI Foundry Agents?

Technically it should. All the calls should proxy to the endpoint mentioned in the connection.

[graemefoster]

Tested - it does work. Wonder if that would technically be supported.

[karpikpl]

so changing the endpoint and targeting APIM worked?

I was going to suggest creating N project with N capability hosts, rather than switching models

[graemefoster]

Yup changing the endpoint worked - note I'm testing locally and my pattern is:

• change the endpoint of the connected resource to a devtunnel pointing local
• run AICentral locally (a LLM proxy), but technically this should be no different to running APIm
• pass through the bearer token / api-key that the agent service to the backend LLM

Interestingly the Agent Service uses /completions, not /chat/completions. If you pass through a bearer token with your own identity it doesn't work. So there's definitely something special in those headers from the agent-service.

For example - when I use my own Bearer tokens instead of those from Agent Service I get this:
The completion operation does not work with the specified model, gpt-4.1-mini.

[ricky-moore]

We had this same issue and just got this response yesterday:

I hope this email finds you well. I’m reaching out to provide an update on the issue we’ve been working on related to connected agents. After further testing, I was able to reproduce the same error within the enterprise setup for the East US region. To address this, we have a planned change scheduled for rollout that is expected to resolve the issue. This rollout is anticipated to take place next week. In the meantime, I recommend testing the implementation in one of the following regions:

West US
South India
West US 3
Japan East
UK South
Norway East

[ricky-moore]

West US 3 worked for us

[nrobert]

So you managed to use models deployed in an other Azure OpenAI resource, that you connected to your Foundry / Foundry project?

[ricky-moore]

Putting the model in foundry not adding an additional azure OpenAI resource. Thanks, Rick

…

________________________________ From: Nicolas ***@***.***> Sent: Saturday, June 28, 2025 7:33:22 AM To: azure-ai-foundry/discussions ***@***.***> Cc: Rick Moore ***@***.***>; Comment ***@***.***> Subject: Re: [azure-ai-foundry/discussions] Agents - Cannot use model deployments from connected resources? (Discussion #59) Warning: This is an external email, please use caution clicking links or opening attachments. So you managed to use models deployed in an other Azure OpenAI resource, that you connected to your Foundry / Foundry project? — Reply to this email directly, view it on GitHub<#59 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BEPQVQ4GSOPCMNYL5WL72KT3FZ4QFAVCNFSM6AAAAAB7T7YUYWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNRQGQ2TONA>. You are receiving this because you commented.Message ID: ***@***.***>

[nrobert]

It looks like that was not the initial question