Secure network agent via terraform to support isolated GA agents.

[trentderby]

Introduction

Adding this so it will hopefully help some people who are more terraform-oriented to create secure network agents using AI foundry.

Basically, the terraform equivalent of this:

https://learn.microsoft.com/en-us/azure/ai-services/agents/how-to/virtual-networks

Some caveats -

This is a translation of the MS provided bicep over to terraform - a lot of the AzureRM providers do not yet exist - and the naming of some of them might confuse things a little - for this - the AI foundry project is the project I will be working off - so no AI hub involved - the existing supported terraform AzureRM provider for AI Foundry project creates a HUB based project, and network isolated agents are not yet supported for this - and are still not in GA at time of writing.

For the most part, AzAPI has been used to plug gaps, but it's functional and will work while we await the azurerm equivalents to come live in a few weeks.

Features and Screenshots

Creates a secure AI foundry + project with all necessary resources isolated and connected.

Technical Details

Terraform - using AzureRM and Azapi resources - plus PowerShell for cosmos DB permissions.

A lot of improvements can be made - this has been deployed a few times in our environment - would like to get the cosmos permissions also via TF, but palming that off to a PowerShell for now.

IAM permissions - by and large, you will be required to give the MIDs of each resource permission to other resources - please message if stuck on any of this.

PRIVATE ENDPOINTS:

Assumptions - awareness of how to create a private endpoint to secure resources - sample here given for cosmos DB, other subresources required are - account, Sql, vault, blob, file (not yet supported) and searchservice.

zone names required: privatelink.services.ai.azure.com, privatelink.cognitiveservices.azure.com, privatelink.vaultcore.azure.net, privatelink.file.core.windows.net, privatelink.blob.core.windows.net, privatelink.search.windows.net, privatelink.openai.azure.com

# Create the Private Endpoint for cosmos DB
resource "azurerm_private_endpoint" "pecosmosdb" { 
  name                = var.cosmosdb_private_endpoint_name
  depends_on = [azurerm_cosmosdb_account.cmosdb]
  location            = var.location
  resource_group_name = var.resource_group_name
  subnet_id           = azurerm_subnet.subnetmain.id

private_service_connection {
  name                           = "PE-PSC-${var.cosmosdb_name}"
  private_connection_resource_id = azurerm_cosmosdb_account.cmosdb.id
  is_manual_connection           = false
  subresource_names = [ "Sql" ]
  }

private_dns_zone_group {
  name                 = "cosmosdbdnszonegroup"
  private_dns_zone_ids = [azurerm_private_dns_zone.cosmosdb_dns_zone.id]
 }
}

# Create cosmos DB Private DNS Zone Virtual Network Link
resource "azurerm_private_dns_zone_virtual_network_link" "cosmosdbdnszvnetlink" {
  name                  = var.cosmosdb_private_zone_vl_name
  resource_group_name   = var.resource_group_name
  private_dns_zone_name = azurerm_private_dns_zone.cosmosdb_dns_zone.name
  virtual_network_id    = azurerm_virtual_network.vnetmain.id
  tags = var.tags
}

# Create Private DNS Zone for cosmos DB
resource "azurerm_private_dns_zone" "cosmosdb_dns_zone" {
  name                = var.cosmosdb_private_dns_zone_name
  resource_group_name = var.resource_group_name
}

NETWORKING:

Outside of all resources being behind PEs (as above) - the network itself will need to have a specific addressing - B or C type network - the agent will not work unless configured this way - here is a sample of vnet and subnet config - VNET - then 2 subnets, one for private endpoints and resources, the other dedicated to agents:

# Create the VNET for Private Endpoint
resource "azurerm_virtual_network" "vnetmain" {
  name                = var.virtual_network_name
  depends_on = [var.resource_group_name]
  address_space       = [var.vnet_address_space]
  location            = var.location
  resource_group_name = var.resource_group_name
  tags = var.tags
}

# Create the Subnet for Private Endpoint
resource "azurerm_subnet" "subnetmain" {
  name                 = var.subnet_name
  depends_on = [azurerm_virtual_network.vnetmain]
  resource_group_name  = var.resource_group_name
  virtual_network_name = var.virtual_network_name
  address_prefixes     = [var.subnet_address_space]
  private_link_service_network_policies_enabled = true
  private_endpoint_network_policies = "Enabled"
  service_endpoints = [
    "Microsoft.Storage",
    "Microsoft.CognitiveServices",
    "Microsoft.Web",
    "Microsoft.KeyVault",
    "Microsoft.Sql",
    "Microsoft.EventHub",
    "Microsoft.ServiceBus",
    "Microsoft.AzureActiveDirectory"
  ]
}

# Create the Subnet for AI Agents
resource "azurerm_subnet" "subnetagents" {
  name                 = var.agentsubnet_name
  depends_on = [azurerm_virtual_network.vnetmain]
  resource_group_name  = var.resource_group_name
  virtual_network_name = var.virtual_network_name
  address_prefixes     = [var.agentsubnet_address_space]
  delegation {
    name = "delegation"
    service_delegation {
      name = "Microsoft.App/environments"
      actions = [
        "Microsoft.Network/virtualNetworks/subnets/join/action",
      ]
    }
  }
}

Variables for network address space (smaller than ms reccomendation for our environment, has to be a class B or C network):

vnet_address_space   = "172.16.0.0/21"
subnet_address_space = "172.16.0.0/22"
agentsubnet_address_space = "172.16.4.0/22"

AI FOUNDRY RESOURCES/CONNECTIONS:

The main "secret sauce" of the BICEP - is the capability host - that is the key to having all of this work correctly - please ensure all resources are in the same region - please note you will also have to run a az cli to give cosmos DB permissions over - ill be working on getting this into TF.

#################################
##        AI-FOUNDRY           ##
#################################

# Requires - Key Vault, Storage account, Cosmos DB, AI Search, AI Foundry and AI Foundry Project (new type from 19/05/2025 ), Application Insights
# Per resource -  Private DNS Zone, Private Endpoint, Private DNS Zone Virtual Network Link

#Key Vault for AI Foundry Project - not required but kept for any potential secrets etc..... 

resource "azurerm_key_vault" "keyvault" {
 
  name                      = var.keyvaultname
  location                  = var.location
  resource_group_name       = var.resource_group_name
  tenant_id                 = data.azurerm_client_config.current.tenant_id
  sku_name                  = "standard"
  enable_rbac_authorization = true
  enabled_for_deployment          = false
  enabled_for_disk_encryption     = true
  enabled_for_template_deployment = false
  soft_delete_retention_days      = 90
  purge_protection_enabled        = true
  public_network_access_enabled   = false
  tags                            = var.tags
  network_acls {
    default_action = "Deny"
    bypass         = "AzureServices"
  }
}

# Create AI Search Service for AI foundry Project
resource "azurerm_search_service" "ai-search-service" {
  name                = var.ai_search_name
  resource_group_name = var.resource_group_name
  location            = azurerm_resource_group.rg.location
  sku                 = var.ai_search_sku
  tags                          = var.tags
  identity {
    type = "SystemAssigned"
  }
  local_authentication_enabled = true
  authentication_failure_mode  = "http403"
  public_network_access_enabled = false
  network_rule_bypass_option = "AzureServices"
  semantic_search_sku = "standard"
}


#Storage account for AI Foundry Project
resource "azurerm_storage_account" "storageaccount" {
  name                = var.storageaccountname
  resource_group_name = var.resource_group_name
  location                 = var.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  allow_nested_items_to_be_public = false
  default_to_oauth_authentication = true
  tags = var.tags
  network_rules {
    default_action             = "Deny"
  }
}

# Cosmos DB for AI Foundry Project
resource "azurerm_cosmosdb_account" "cmosdb" {
  name                = var.cosmosdb_name
  location            = var.location
  tags                = var.tags
  resource_group_name = var.resource_group_name
  offer_type          = "Standard"
  kind                = "GlobalDocumentDB"

  consistency_policy {
    consistency_level       = "Session"
  }

  geo_location {
    location          = var.location
    failover_priority = 0
    zone_redundant = false
  }
  identity {
    type = "SystemAssigned"
  }
  automatic_failover_enabled = false
  public_network_access_enabled = false
  free_tier_enabled = false
  local_authentication_disabled = true
  
}

# Assign permissions to Cosmos DB for AI Foundry Project for each container using az cli commands!! 



# AI Foundry Resource - using AzAPI provider to get the latest features and network injection support - var.networkinjection is literally #"true" - but required to pass in a true value - I'm sure there is other ways to achieve this. 


resource "azapi_resource" "aiservicesresource" {
  type = "Microsoft.CognitiveServices/accounts@2025-04-01-preview"
  name = var.ai_foundry_name
  location = var.ai_project_location
  parent_id = azurerm_resource_group.rg.id
  

  identity {
    type = "SystemAssigned"
  }

body = {
    name = var.ai_foundry_name
    properties = {
      allowProjectManagement = true
      customSubDomainName = var.ai_foundry_name
      publicNetworkAccess = "disabled"
      networkAcls = {
        defaultAction = "Allow"
        bypass = "AzureServices"
      }
      networkInjections = var.network_injection == "true" ? [ #ensure you have a var for this or can pass "true" in
      {
        scenario = "Agent"
        subnetArmId = azurerm_subnet.subnetagents.id
        useMicrosoftManagedNetwork = false
      }
      ] : null
      disablelocalAuth = false
      apiProperties = {
      }
    }
    kind = "AIServices"
    sku = {
      name = var.ai_services_sku
    }
  }
  response_export_values = ["*"]
  schema_validation_enabled = false
}



# AI Foundry Project Resource - using AzAPI provider to get the latest features and network injection support

resource "azapi_resource" "project" {
  type = "Microsoft.CognitiveServices/accounts/projects@2025-04-01-preview"
  name = var.ai_project_name
  location = var.ai_project_location
  parent_id = azapi_resource.aiservicesresource.id

  identity {
    type = "SystemAssigned"
  }

  body = {
    properties = {
      description = "Your description"
      friendlyName = "Your friendly name"
    }
  }
  schema_validation_enabled = false
}

# Project connections #

# Cosmos DB Connection for AI Foundry Project
resource "azapi_resource" "aiproject_cosmosdb_connection" {
  type = "Microsoft.CognitiveServices/accounts/connections@2025-04-01-preview"
  name = "${azurerm_cosmosdb_account.cmosdb.name}-aip-connection"
  parent_id = azapi_resource.aiservicesresource.id

  body = {
    properties = {
      category = "CosmosDB",
      target = azurerm_cosmosdb_account.cmosdb.endpoint,
      authType = "AAD",
      isSharedToAll = true,
      metadata = {
        ApiType = "Azure",
        ResourceId = azurerm_cosmosdb_account.cmosdb.id,
        location = azurerm_cosmosdb_account.cmosdb.location,
      }
    }
  }
  response_export_values = ["*"]
}

# Storage Account Connection for AI Foundry Project
resource "azapi_resource" "aiproject_storageaccount_connection" {
  type = "Microsoft.CognitiveServices/accounts/connections@2025-04-01-preview"
  name = var.storageaccountname
  parent_id = azapi_resource.aiservicesresource.id

  body = {
    properties = {
      category = "AzureStorageAccount",
      target = azurerm_storage_account.storageaccount.primary_blob_endpoint,
      authType = "AAD",
      isSharedToAll = true,
      metadata = {
        ApiType = "Azure",
        ResourceId = azurerm_storage_account.storageaccount.id,
        location = azurerm_storage_account.storageaccount.location
      }
    }
  }
  response_export_values = ["*"]
}
# AI Search Connection for AI Foundry Project
resource "azapi_resource" "aiproject_search_connection" {
  type = "Microsoft.CognitiveServices/accounts/connections@2025-04-01-preview"
  name = "${azurerm_search_service.ai-search-service.name}-aip-connection"
  parent_id = azapi_resource.aiservicesresource.id

  body = {
    properties = {
      category = "CognitiveSearch",
      target = "https://${azurerm_search_service.ai-search-service.name}.search.windows.net",
      authType = "AAD",
      isSharedToAll = true,
      metadata = {
        ApiType = "Azure",
        ResourceId = azurerm_search_service.ai-search-service.id,
        location = azurerm_search_service.ai-search-service.location
      }
    }
  }
  response_export_values = ["*"]
}

# AI Foundry Project Capability Host
resource "azapi_resource" "capability_host_aiproject" {
  type      = "Microsoft.CognitiveServices/accounts/projects/capabilityHosts@2025-04-01-preview"
  parent_id = azapi_resource.project.id
  name      = var.capabilityhostprojectname

  body = ({
    properties = {
      capabilityHostKind       = "Agents"
      vectorStoreConnections   = [azapi_resource.aiproject_search_connection.name]
      storageConnections       = [azapi_resource.aiproject_storageaccount_connection.name]
      threadStorageConnections = [azapi_resource.aiproject_cosmosdb_connection.name]
    }
  })
  depends_on = [ azurerm_role_assignment.aiproject_cosmosdb_operator, azurerm_role_assignment.aiproject_storageaccount_file_data_privileged_contributor ]
}

resource "azurerm_application_insights" "default" {
  name                = var.application_insights_name
  location            = azurerm_resource_group.rg.location
  resource_group_name = azurerm_resource_group.rg.name
  application_type    = "web"
  tags = var.tags
  lifecycle {
      ignore_changes   = [workspace_id]
   }
}

COSMOS DB Script

# needs - Install-Module -Name Az.CosmosDB

# Common variables
$resourceGroupName = "Your RG name"
$accountName = "Cosmos DB account name"
$principalId = "xxxxxxxxxx" # MID of project, or ID of group who needs access
$roleDefinitionId = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$resourceGroupName/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002"
$subscriptionId = "Your sub ID"
$projectWorkspaceId = "Your project workspace ID" # Project workspace ID, replace with your actual workspace ID as it shows in data explorer/JSON 

# Set the context to the correct subscription
Set-AzContext -SubscriptionId $subscriptionId

# Define container scopes
$scopes = @(
    "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$accountName/dbs/enterprise_memory/colls/$projectWorkspaceId-system-thread-message-store",
    "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$accountName/dbs/enterprise_memory/colls/$projectWorkspaceId-thread-message-store",
    "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.DocumentDB/databaseAccounts/$accountName/dbs/enterprise_memory/colls/$projectWorkspaceId-agent-entity-store"
)

# Create role assignments for each scope
foreach ($scope in $scopes) {
    try {
        Write-Host "Creating role assignment for scope: $scope"
        
        New-AzCosmosDBSqlRoleAssignment -AccountName $accountName `
            -ResourceGroupName $resourceGroupName `
            -RoleDefinitionId $roleDefinitionId `
            -PrincipalId $principalId `
            -Scope $scope
            
        Write-Host "✓ Successfully created role assignment for: $scope" -ForegroundColor Green
    }
    catch {
        Write-Host "✗ Failed to create role assignment for: $scope" -ForegroundColor Red
        Write-Host "Error: $($_.Exception.Message)" -ForegroundColor Red
    }
}

Write-Host "`nRole assignment creation completed." -ForegroundColor Yellow

IAM Permissions

I made a table out of the IAM we have set up for this - here is a snippet of one role:

# Permissions for AI Project to access cosmos DB
resource "azurerm_role_assignment" "aiproject_cosmosdb_operator" {
    scope                = azurerm_cosmosdb_account.cmosdb.id
    role_definition_name = "Cosmos DB Operator"
    principal_id         = azapi_resource.project.identity[0].principal_id
}

IAM would be too large to post in here, but I have got an output in a table format:

IAM Permissions Overview Table

Search Service Permissions

Principal	Principal Type	Role	Purpose
AI Foundry Project	Managed Identity	Search Index Data Contributor	Project contributes to search indexes
AI Foundry Project	Managed Identity	Search Service Contributor	Project manages search service
AI Foundry	Managed Identity	Search Index Data Contributor	AI Foundry contributes to search indexes
AI Foundry	Managed Identity	Search Index Data Reader	AI Foundry reads search indexes
AI Foundry	Managed Identity	Search Service Contributor	AI Foundry manages search service

AI Foundry Permissions

Principal	Principal Type	Role	Purpose
AI Foundry	Managed Identity	Azure AI Developer	Self-reference for AI Foundry
AI Foundry Project	Managed Identity	Azure AI Developer	Project develops with AI Foundry
Search Service	Managed Identity	Cognitive Services Contributor	Search manages AI Foundry
Search Service	Managed Identity	Cognitive Services OpenAI Contributor	Search contributes to AI Foundry

Cross-Service AI Permissions

Principal	Principal Type	Role	Scope	Purpose
AI Foundry	Managed Identity	Azure AI Developer	AI Project	AI Foundry with Project
AI Project	Managed Identity	Azure AI Developer	AI Foundry	Project develops with AI Foundry

Storage Account Permissions

Principal	Principal Type	Role	Purpose
AI Foundry Project	Managed Identity	Storage File Data Privileged contributor	Project manages file storage
AI Foundry	Managed Identity	Storage Blob Data Contributor	AI Foundry contributes to blob storage
AI Project	Managed Identity	Storage Blob Data Contributor	Project contributes to blob storage
Search Service	Managed Identity	Storage Blob Data Contributor	Search contributes to blob storage

Private Endpoint Permissions

Principal	Principal Type	Role	Scope	Purpose
AI Foundry	Managed Identity	Reader	Storage Account File PE	AI Foundry reads file PE
AI Foundry	Managed Identity	Reader	Storage Account Blob PE	AI Foundry reads blob PE
AI Foundry Project	Managed Identity	Reader	Storage Account File PE	Project reads file PE
AI Foundry Project	Managed Identity	Reader	Storage Account Blob PE	Project reads blob PE

Application Insights Permissions

Principal	Principal Type	Role	Purpose
AI Project	Managed Identity	Contributor	Project manages App Insights

Cosmos DB Permissions

Principal	Principal Type	Role	Purpose
AI Project	Managed Identity	Cosmos DB Operator	Project operates Cosmos DB

Challenges and Solutions

Usual challenges with the DNS endpoints and pointing to a 172.x.x.x network. have your networking team on standby to ensure all resources can resolve to each other.

A lot of IAM permissions required for inter resource. MIDs are used for everything in this project and work.

This is by no means fully complete and a lot of improvements to come - but I'm hoping this can get some of you started using secure AI foundry via terraform.

[jackirvine97]

Very cool - thanks for sharing

[amynic]

Thanks for sharing @trentderby 🎉 - this is a great show and tell to support people taking Azure AI Foundry Agents from POC to production - especially as the service went GA at Microsoft Build on 19th May

@guygregory will likely find this interesting for partners and customers

[revodavid]

Thanks so much for sharing -- this will be super useful for any AI Foundry / Terraform user.

[trentderby]

This is now in the main MS repo now - so I would recommend using that:

https://github.com/azure-ai-foundry/foundry-samples/blob/main/samples/microsoft/infrastructure-setup-terraform/15a-private-network-standard-agent-setup/code/main.tf