I am getting confused about Azure Agents. Is it in Azure OpenAI Service or in Azure AI services

[Hyunduck-Hwang]

I have managed to create Agents in Azure AI Foundry and used it few days now. I am still confused where it belongs. I need to know exactly how it works so that I can decide which products should be used in my project. Let me drop some images.



As you can see, resource open-nah564626892761 is Azure OpenAI Service in the 2nd image. But it is also Azure AI services in the 1st image.
Is it possible that I create Azure agent without Azure AI services? Or DO I need both services?
This is so frustrating....RBAC role things were also difficult to understand and use.....

[timothywarner]

I will add to the discussion that I'm concerned MSFT is using "agent" in an unhelpful way. For example, the "agents" I see in M365 Copilot are essentially ChatGPT assistants, heavily feature-restricted, correct?

Whereas agentic AI agents have additional permissions to act autonomously and call tools from MCP or OpenAI's implementation of same.

Then we have "chat participant," at least in the VS Code Copilot extension API. These can be agentic or non agentic.

So yeah, as OP said: What is an 'agent' in Azure AI Foundry? :)

With love and respect,
Tim Warner

[amynic]

Thanks Tim for the thoughtful overview of the many AI integrations Microsoft has to offer. I have reached out to some colleagues for Microsoft level guidance around Agents but keeping it closer to home and answering: What is an 'agent' in Azure AI Foundry?

Within Azure AI Foundry, an AI Agent acts as a "smart" microservice that can be used to answer questions (RAG), perform actions, or completely automate workflows. It achieves this by combining the power of generative AI models with tools that allow it to access and interact with real-world data sources. (https://learn.microsoft.com/en-us/azure/ai-services/agents/overview)

[amynic]

Thanks for sharing your question @Hyunduck-Hwang - some thoughts below that could help you understand the different elements of your agent architecture

At the top level you have your Azure AI Foundry Hub and Project Model: https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/ai-resources, Hub: for a team to govern security, connectivity, and computing resources across playgrounds and projects. Once a hub is created, developers can create projects from it and access shared company resources without needing an IT administrator's repeated help. Project workspaces that are created using a hub inherit the same security settings and shared resource access. Teams can create project workspaces as needed to organize their work, isolate data, and/or restrict access.

Inside of Azure AI Foundry you have lots of connected services that help you create your Gen AI apps - things such as Azure AI Services instances (Our AI services include things like prebuilt Speech API, Vision, Translator, Document intelligence), in our product hierarchy today Azure Open AI Service is considered an Azure AI Service, read more here: https://learn.microsoft.com/en-us/azure/ai-foundry/azure-openai-in-azure-ai-foundry

When you create a hub in Azure AI Foundry through the Foundry portal, the default setup requires connecting to an Azure AI Services instance. This is because the hub is designed to manage access to all models and services, including those in the model catalog.

However, there are a few important nuances:

• Default behavior: When creating a hub or project in the Azure AI Foundry portal, a new Azure AI Services resource is created by default.
• Manual setup alternative: If you want to avoid this automatic connection, you can: Use Azure CLI, Bicep, or Terraform to create a custom setup.
  Using external models: You can still use models from the Model Catalog (like Meta or Hugging Face) without relying on Azure AI Services, but you’ll need to manage the deployment and inference endpoints yourself or through other Azure compute resources (e.g., Azure Kubernetes Service, Azure Machine Learning). - this doc has a nice diagram at the top: https://learn.microsoft.com/en-us/azure/ai-foundry/model-inference/how-to/quickstart-create-resources?pivots=programming-language-bicep

Summary
✅ Yes, you can avoid connecting to Azure AI Services, but not through the default UI flow. AI services helps you deploy and manage any models from the Azure AI Foundry Model Catalog
🔧 You’ll need to use infrastructure-as-code tools or manual configuration to bypass the default AI Services connection.

Hope some of these resources help!

[Hyunduck-Hwang]

First, Thank you for the answer. So can I make Azure agent with only Azure OpenAI service by using Azure CLI and Bicep? I do not need external models for now.

[amynic]

If you don't need external models and want to build an agent using the Azure AI Agent service with Azure Open AI models - you will have an Azure AI Foundry instance and an Azure Open AI Service instance in your architecture - this is then a fully managed architecture via PaaS services